-- Logs begin at Thu 2022-05-26 14:16:01 UTC, end at Thu 2022-05-26 15:29:43 UTC. -- May 26 14:53:29.479490 dv-cinder-788367-11-iscsi systemd[1]: Starting Devstack devstack@keystone.service... May 26 14:53:29.487622 dv-cinder-788367-11-iscsi devstack@keystone.service[80159]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini May 26 14:53:29.490332 dv-cinder-788367-11-iscsi devstack@keystone.service[80159]: *** Starting uWSGI 2.0.18-debian (64bit) on [Thu May 26 14:53:29 2022] *** May 26 14:53:29.490476 dv-cinder-788367-11-iscsi devstack@keystone.service[80159]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 May 26 14:53:29.490582 dv-cinder-788367-11-iscsi devstack@keystone.service[80159]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 May 26 14:53:29.490683 dv-cinder-788367-11-iscsi devstack@keystone.service[80159]: nodename: dv-cinder-788367-11-iscsi May 26 14:53:29.490783 dv-cinder-788367-11-iscsi devstack@keystone.service[80159]: machine: x86_64 May 26 14:53:29.490884 dv-cinder-788367-11-iscsi devstack@keystone.service[80159]: clock source: unix May 26 14:53:29.490983 dv-cinder-788367-11-iscsi devstack@keystone.service[80159]: pcre jit disabled May 26 14:53:29.491088 dv-cinder-788367-11-iscsi devstack@keystone.service[80159]: detected number of CPU cores: 4 May 26 14:53:29.491187 dv-cinder-788367-11-iscsi devstack@keystone.service[80159]: current working directory: / May 26 14:53:29.491280 dv-cinder-788367-11-iscsi devstack@keystone.service[80159]: detected binary path: /usr/bin/uwsgi-core May 26 14:53:29.491386 dv-cinder-788367-11-iscsi devstack@keystone.service[80159]: your processes number limit is 47680 May 26 14:53:29.491485 dv-cinder-788367-11-iscsi devstack@keystone.service[80159]: your memory page size is 4096 bytes May 26 14:53:29.491599 dv-cinder-788367-11-iscsi devstack@keystone.service[80159]: detected max file descriptor number: 2048 May 26 14:53:29.491599 dv-cinder-788367-11-iscsi devstack@keystone.service[80159]: lock engine: pthread robust mutexes May 26 14:53:29.491798 dv-cinder-788367-11-iscsi devstack@keystone.service[80159]: thunder lock: enabled May 26 14:53:29.491938 dv-cinder-788367-11-iscsi devstack@keystone.service[80159]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 May 26 14:53:29.492029 dv-cinder-788367-11-iscsi devstack@keystone.service[80159]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] May 26 14:53:29.537118 dv-cinder-788367-11-iscsi devstack@keystone.service[80159]: Python main interpreter initialized at 0x55e9034ef9c0 May 26 14:53:29.537118 dv-cinder-788367-11-iscsi devstack@keystone.service[80159]: python threads support enabled May 26 14:53:29.537118 dv-cinder-788367-11-iscsi devstack@keystone.service[80159]: your server socket listen backlog is limited to 100 connections May 26 14:53:29.537118 dv-cinder-788367-11-iscsi devstack@keystone.service[80159]: your mercy for graceful operations on workers is 90 seconds May 26 14:53:29.537470 dv-cinder-788367-11-iscsi devstack@keystone.service[80159]: mapped 403077 bytes (393 KB) for 2 cores May 26 14:53:29.537552 dv-cinder-788367-11-iscsi devstack@keystone.service[80159]: *** Operational MODE: preforking *** May 26 14:53:29.537836 dv-cinder-788367-11-iscsi devstack@keystone.service[80159]: *** uWSGI is running in multiple interpreter mode *** May 26 14:53:29.537836 dv-cinder-788367-11-iscsi devstack@keystone.service[80159]: spawned uWSGI master process (pid: 80159) May 26 14:53:29.538037 dv-cinder-788367-11-iscsi systemd[1]: Started Devstack devstack@keystone.service. May 26 14:53:29.538951 dv-cinder-788367-11-iscsi devstack@keystone.service[80159]: spawned uWSGI worker 1 (pid: 80167, cores: 1) May 26 14:53:29.538951 dv-cinder-788367-11-iscsi devstack@keystone.service[80159]: spawned uWSGI worker 2 (pid: 80168, cores: 1) May 26 14:53:29.539100 dv-cinder-788367-11-iscsi devstack@keystone.service[80159]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... May 26 14:53:35.228965 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 26 14:53:35.228965 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: from cryptography.utils import int_from_bytes May 26 14:53:35.229780 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 26 14:53:35.229780 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: from cryptography.utils import int_from_bytes May 26 14:53:35.250755 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 26 14:53:35.250755 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: from cryptography.utils import int_from_bytes May 26 14:53:35.251328 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 26 14:53:35.251328 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: from cryptography.utils import int_from_bytes May 26 14:53:35.280202 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} May 26 14:53:35.280379 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} May 26 14:53:35.280557 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] command line args: [] {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} May 26 14:53:35.280820 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} May 26 14:53:35.280992 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] ================================================================================ {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} May 26 14:53:35.281247 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] admin_token = **** {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 14:53:35.281505 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] config_dir = [] {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 14:53:35.281692 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 14:53:35.281870 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] config_source = [] {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 14:53:35.282043 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] debug = True {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 14:53:35.282248 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 14:53:35.282616 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 14:53:35.282800 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 14:53:35.283050 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 14:53:35.283294 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 14:53:35.283520 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] list_limit = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 14:53:35.283696 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] log_config_append = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 14:53:35.283869 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 14:53:35.284041 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] log_dir = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 14:53:35.284214 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] log_file = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 14:53:35.284447 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 14:53:35.284692 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 14:53:35.284928 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 14:53:35.285179 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 14:53:35.285355 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 14:53:35.285569 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 14:53:35.285813 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 14:53:35.286054 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 14:53:35.286285 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 14:53:35.286529 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 14:53:35.286741 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 14:53:35.286970 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 14:53:35.287213 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 14:53:35.287455 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 14:53:35.287725 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 14:53:35.288210 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.46/identity {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 14:53:35.288388 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] publish_errors = False {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 14:53:35.288615 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 14:53:35.288840 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 14:53:35.289013 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 14:53:35.289254 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 14:53:35.289503 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 14:53:35.289738 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] standard_threads = False {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 14:53:35.289966 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 14:53:35.290204 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 14:53:35.290378 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 14:53:35.290550 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] use_journal = False {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 14:53:35.290721 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] use_json = False {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 14:53:35.290891 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] use_stderr = False {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 14:53:35.291063 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] use_syslog = False {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 14:53:35.291289 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 14:53:35.291543 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.291776 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.292032 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.292267 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.292510 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.292748 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.292990 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.293222 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.293485 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.294278 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.294527 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.294758 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.294999 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.295233 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.295476 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.295718 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.295949 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.296196 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.296443 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.296683 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.296915 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.297162 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.297925 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.297925 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.297925 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.298232 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.298357 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.298582 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.298799 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} May 26 14:53:35.298885 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.299016 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} May 26 14:53:35.299098 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.299176 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] command line args: [] {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} May 26 14:53:35.299272 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.299431 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} May 26 14:53:35.299514 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.299607 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ================================================================================ {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} May 26 14:53:35.299696 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.299855 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] admin_token = **** {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 14:53:35.299932 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.300095 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] config_dir = [] {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 14:53:35.300172 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.300278 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 14:53:35.300367 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.300461 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] config_source = [] {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 14:53:35.300611 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.300688 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] debug = True {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 14:53:35.300837 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.300921 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 14:53:35.301069 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 14:53:35.301146 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.301245 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 14:53:35.301323 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.301535 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.301627 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 14:53:35.301765 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.301866 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 14:53:35.301992 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.302094 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] list_limit = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 14:53:35.302218 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.302303 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] log_config_append = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 14:53:35.302433 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 14:53:35.302509 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.302607 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] log_dir = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 14:53:35.302684 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.302781 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] log_file = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 14:53:35.302933 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.303007 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 14:53:35.303116 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.303250 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 14:53:35.303347 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.303491 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 14:53:35.303577 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.303733 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 14:53:35.303818 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.303895 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 14:53:35.304027 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.304114 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 14:53:35.304253 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.304335 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 14:53:35.304487 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.304564 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 14:53:35.304719 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.304809 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 14:53:35.304950 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.305033 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 14:53:35.305181 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.305264 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 14:53:35.305407 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 14:53:35.305519 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.305660 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.305742 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 14:53:35.305904 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.305981 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 14:53:35.306136 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.306221 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 14:53:35.306370 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.306597 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.306681 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.46/identity {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 14:53:35.306816 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] publish_errors = False {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 14:53:35.306901 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.307043 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 14:53:35.307120 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.307269 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 14:53:35.307347 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.307492 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 14:53:35.307567 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.307671 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 14:53:35.307749 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.307901 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 14:53:35.307978 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.308129 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] standard_threads = False {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 14:53:35.308207 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.308343 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 14:53:35.308427 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.308580 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 14:53:35.308657 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.308756 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 14:53:35.308883 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.308965 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] use_journal = False {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 14:53:35.309091 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] use_json = False {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 14:53:35.309167 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.309286 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] use_stderr = False {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 14:53:35.309996 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.310075 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] use_syslog = False {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 14:53:35.310147 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.310216 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 26 14:53:35.310365 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.310443 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.310599 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.310677 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.310836 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.310919 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.311070 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.311148 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.311304 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.311382 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.311535 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.311643 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.311766 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.311870 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.311997 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.312110 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.312252 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.312363 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.312474 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.312572 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.312699 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.312777 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.312930 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.313011 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.313166 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.313255 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.313398 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.313526 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.313655 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.313739 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.313885 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.313975 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.314110 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.314197 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.314334 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.314463 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.314571 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.314658 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.314802 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.314889 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.315026 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.315149 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.315263 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.315354 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.315497 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.315576 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.315740 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.315822 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.315972 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.316092 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.316199 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.316285 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.316428 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.316549 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.316655 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.316783 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.316888 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.316974 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] auth.external = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.317113 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.317191 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.317362 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.317438 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.317611 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.317692 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.317838 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.317917 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] auth.password = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.318075 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] auth.token = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.318152 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.318309 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.318389 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.318540 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.318619 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.318795 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.318876 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.319007 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.319134 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.319242 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.319339 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.319482 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.319607 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.319772 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.319894 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.319971 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] credential.caching = True {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.320049 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.320157 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.320236 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.320408 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.320532 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.320643 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.320761 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.320865 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.320930 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.321094 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.321173 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.321321 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.321398 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.321569 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.321649 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.321804 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.321884 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.322037 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.322116 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.322281 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.322360 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.322507 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.322594 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.322743 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.322886 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.322963 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.323035 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.323205 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.323340 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.323457 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.323579 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.323687 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.323824 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] federation.caching = True {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.323922 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.324045 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.324160 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.324281 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.324396 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.324509 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.324632 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.324755 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.324866 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.324981 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.325073 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.325148 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.325242 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.325330 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.325531 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.325617 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.325724 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.325838 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.325969 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.326056 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.326235 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.326339 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.326472 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.326552 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] identity.caching = True {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.326713 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.326790 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.326953 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.327038 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.327186 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.327345 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.327423 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.327511 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.327641 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.327767 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.327879 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.327959 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.328115 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.328194 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.328350 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.328437 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.328589 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.328674 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.328828 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.328916 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.329059 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.329156 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.329287 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.329367 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.329540 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.329622 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.329777 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.329891 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.330016 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.330093 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.330260 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.330339 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.330495 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.330583 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.330760 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.330838 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.330999 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.331121 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.331232 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.331321 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.331461 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.331547 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.331696 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.331792 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.331924 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.332048 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.332157 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.332238 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.332394 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.332474 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.332628 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.332715 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.332871 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.332956 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.333098 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.333177 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.333336 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.333548 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.333627 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.333688 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.333824 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.333905 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.334065 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.334153 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.334308 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.334387 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.334533 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.334612 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.334759 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] auth.external = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.334845 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.334985 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.335084 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.335215 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.335294 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.335446 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.335534 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.335684 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.335769 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] auth.password = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.335909 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] auth.token = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.335988 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.336140 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.336225 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.336370 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.336454 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.336611 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.336690 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.336836 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.336914 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.337058 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.337135 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.337301 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.337379 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.337564 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.337644 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.337796 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.337874 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] credential.caching = True {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.338027 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.338106 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.338259 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.338340 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.338484 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] ldap.user = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.338562 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.338748 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.338881 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.339014 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.339137 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.339263 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.339386 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.339509 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.339632 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.339764 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.339889 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.339889 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.340121 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.340121 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.340376 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.340507 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.340651 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.340777 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.340901 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.341025 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.341148 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.341282 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.341407 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.341564 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.341696 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.341824 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.341951 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] federation.caching = True {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.342096 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.342224 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.342350 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.342484 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.342605 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.342732 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.342867 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.343003 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.343131 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.343258 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.343385 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.343512 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.343646 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.343785 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.343913 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.344040 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.344167 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.344293 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.344433 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.344562 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.344689 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.344815 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.344943 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.345070 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] identity.caching = True {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.345204 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.345331 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.345472 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.345608 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.345733 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.345856 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.345990 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.346115 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.346239 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.346362 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.346491 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.346615 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.346745 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.346869 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.346998 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.347122 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.347254 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.347379 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.347510 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.347634 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.347758 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.347880 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.348016 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] resource.caching = True {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.348141 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.348272 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.348396 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.348520 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.348643 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.348767 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.348890 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.349030 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.349156 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.349279 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.349408 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.349576 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.349714 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.349843 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.349970 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.350098 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.350224 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.350350 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.350484 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.350566 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] role.caching = True {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.350693 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.350817 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] role.driver = sql {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.350901 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.350963 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.351031 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.351093 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.351152 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.351212 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.351271 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.351271 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.351492 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.351492 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.351602 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.351662 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.351720 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.351779 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.351779 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.352110 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.352291 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.352421 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.352563 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.352563 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.352807 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.352938 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.353066 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.353232 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.353365 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.353463 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.353608 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.353608 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.353806 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.353926 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.354049 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.354170 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.354261 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.354325 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.354446 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.354568 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.354642 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.354755 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.354887 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.355020 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.355145 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.355268 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.355391 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.355518 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.355657 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.355741 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.355872 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.355998 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.356081 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.356148 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.356908 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.356992 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.357055 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.357236 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.357366 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.357522 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.357647 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.357730 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.user = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.357893 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.357976 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.358090 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.358220 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.358344 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.358467 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.358590 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.358713 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.358845 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] token.caching = True {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.358936 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.359001 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.359060 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.359119 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.359178 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.359237 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.359303 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.359365 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.359425 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.359483 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.359542 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.359695 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.359838 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.359938 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.360180 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.360341 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.360410 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.360599 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.360675 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.360849 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.360917 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.361081 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.361148 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.361324 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.361400 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.361591 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.361661 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.361816 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.361883 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.362056 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.362136 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.362342 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.362797 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.362797 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] database.connection = **** {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.363037 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.363171 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.363295 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.363419 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.363556 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.363680 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.363815 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.363939 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.364063 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.364186 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.364320 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.364445 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.364445 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.364678 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.364801 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.364924 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.365061 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.365186 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.365318 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.365535 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.366299 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.367606 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.367728 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.367728 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.367898 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.367971 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.368043 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.368124 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.368124 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.368124 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.368290 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.368352 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.368352 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80167) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} May 26 14:53:35.368465 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.368465 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.368465 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] resource.caching = True {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.368465 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.368805 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.369188 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.369635 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.370014 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.370260 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.370510 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.370751 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.370992 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.371289 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] role.caching = True {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.371586 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] role.driver = sql {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.371894 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.372682 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.372933 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.373184 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.373433 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.373709 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.373960 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.374207 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.374454 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.374688 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.374933 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.375179 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.375425 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.375668 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.376096 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.376334 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.376584 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.376829 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.377074 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.377323 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.377583 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.377847 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.378099 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.378336 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.378565 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.378796 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.379024 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.379272 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.379524 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.379769 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.380003 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.380236 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.380463 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.380695 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] token.caching = True {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.381187 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.381430 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.381689 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.381944 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.382188 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.382423 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.382662 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.382901 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.383143 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.383378 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.383612 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.383847 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.384088 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.384331 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.384568 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.384807 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.385096 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.385362 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] database.connection = **** {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.385626 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.385872 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.386106 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.386344 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.386594 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.386810 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.387043 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.387276 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.387559 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.387795 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.388035 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.388268 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.388513 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.388744 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.388984 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.389210 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.389461 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.389699 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 26 14:53:35.389869 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80168) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} May 26 14:53:35.398526 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80167) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 14:53:35.398753 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80167) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 14:53:35.398977 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80167) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 14:53:35.408114 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80167) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 14:53:35.415151 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80168) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 14:53:35.415395 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80168) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 14:53:35.415607 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80168) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 14:53:35.426249 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80168) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 14:53:35.437580 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80167) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 14:53:35.437810 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80167) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 14:53:35.449036 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80168) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 14:53:35.449267 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80168) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 14:53:35.492825 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80167) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 14:53:35.493606 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80168) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 14:53:35.546159 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80167) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 14:53:35.546381 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80167) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 14:53:35.546516 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80168) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 14:53:35.546752 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80168) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 14:53:35.614386 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80167) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 14:53:35.614601 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80168) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 14:53:35.614696 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80167) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 14:53:35.614799 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80167) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 14:53:35.614879 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80168) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 14:53:35.615019 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80167) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 14:53:35.615104 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80168) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 14:53:35.615234 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80167) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 14:53:35.615328 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80168) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 14:53:35.615445 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80168) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 14:53:35.615527 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80167) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 14:53:35.615657 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80168) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 14:53:35.639209 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80168) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 14:53:35.639431 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80168) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 14:53:35.639525 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80167) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 14:53:35.639717 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80167) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 14:53:35.639800 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80168) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 14:53:35.639935 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80168) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 14:53:35.640030 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80167) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 14:53:35.640152 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80168) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 14:53:35.640235 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80167) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 14:53:35.640366 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80167) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 14:53:35.640447 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80168) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 14:53:35.640576 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80168) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 14:53:35.640658 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80167) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 14:53:35.640787 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80167) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 14:53:35.652478 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80167) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 14:53:35.653926 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80168) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 14:53:35.665819 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80167) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 14:53:35.666045 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80167) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 14:53:35.666348 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80167) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 14:53:35.666498 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80168) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 14:53:35.666630 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80167) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 14:53:35.666757 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80168) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 14:53:35.666905 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80167) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 14:53:35.666990 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80168) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 14:53:35.667158 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80167) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 14:53:35.667309 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80168) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 14:53:35.667441 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80167) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 14:53:35.667582 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80168) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 14:53:35.667582 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80168) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 14:53:35.667826 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80168) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 26 14:53:35.686371 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 14:53:35.686571 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 26 14:53:35.686899 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 14:53:35.687027 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 14:53:35.687159 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 26 14:53:35.687289 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 26 14:53:35.687420 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 14:53:35.687548 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 14:53:35.687686 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 14:53:35.687816 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 26 14:53:35.687944 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 14:53:35.688085 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 14:53:35.688213 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 14:53:35.688348 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 14:53:35.688478 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 14:53:35.688685 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 14:53:35.699003 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 14:53:35.699272 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 14:53:35.699534 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 14:53:35.699809 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 14:53:35.702047 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 14:53:35.702320 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 14:53:35.702585 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 14:53:35.702863 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 14:53:35.708546 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80167) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 14:53:35.710964 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80168) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 14:53:35.711837 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80167) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 14:53:35.712113 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 14:53:35.712387 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 14:53:35.712661 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 14:53:35.712935 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 14:53:35.713185 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 14:53:35.713599 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 14:53:35.714477 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 14:53:35.714477 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 14:53:35.714707 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80168) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 14:53:35.714788 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 14:53:35.714851 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 14:53:35.714912 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 14:53:35.714973 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 14:53:35.715114 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 14:53:35.715395 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 14:53:35.715794 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 14:53:35.716061 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 14:53:35.716493 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 14:53:35.716637 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 14:53:35.716858 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 14:53:35.717122 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 14:53:35.740907 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 14:53:35.742648 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80167) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 14:53:35.742965 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 14:53:35.743499 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 14:53:35.745200 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80168) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 14:53:35.745493 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 14:53:35.748668 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80167) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 14:53:35.748940 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 14:53:35.749211 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 14:53:35.751268 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80168) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 14:53:35.751591 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 14:53:35.751866 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 14:53:35.757041 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80167) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 14:53:35.757943 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 14:53:35.759624 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80168) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 14:53:35.759901 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 14:53:35.761804 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80167) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 14:53:35.762208 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 14:53:35.763044 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 14:53:35.763335 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 14:53:35.763609 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 14:53:35.763879 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 14:53:35.764141 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 14:53:35.764401 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 14:53:35.764937 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80168) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 14:53:35.765331 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 14:53:35.765650 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 14:53:35.765932 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 14:53:35.766205 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 14:53:35.766478 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 14:53:35.766739 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 14:53:35.767002 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 14:53:35.783133 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 14:53:35.783403 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 14:53:35.785678 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 14:53:35.785953 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 14:53:35.788061 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80167) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 14:53:35.790611 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80168) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 14:53:35.791714 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 14:53:35.792003 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 14:53:35.794295 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 14:53:35.794586 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 14:53:35.797077 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80167) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 14:53:35.799644 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80168) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 14:53:35.800685 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80167) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 14:53:35.803258 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80168) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 14:53:35.804481 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 14:53:35.804778 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 14:53:35.805058 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 14:53:35.805335 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 14:53:35.805641 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 14:53:35.805914 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 14:53:35.807114 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 14:53:35.807411 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 14:53:35.807699 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 14:53:35.807984 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 14:53:35.808264 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 14:53:35.808552 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 14:53:35.826082 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80167) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 14:53:35.826360 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 14:53:35.826621 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 14:53:35.826877 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 14:53:35.828672 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80168) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 14:53:35.828950 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 14:53:35.829212 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 14:53:35.829488 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 14:53:35.834842 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 14:53:35.836557 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 14:53:35.836823 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 14:53:35.837616 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 14:53:35.839311 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 14:53:35.839587 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 14:53:35.839863 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80167) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 14:53:35.840146 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 14:53:35.840422 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 14:53:35.840738 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 14:53:35.841006 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 14:53:35.842668 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80168) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 14:53:35.842946 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 14:53:35.843229 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 14:53:35.843506 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 14:53:35.843776 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 14:53:35.854450 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80167) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 14:53:35.854725 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 14:53:35.855000 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 14:53:35.855271 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 14:53:35.855551 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 14:53:35.855824 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 14:53:35.856100 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 14:53:35.857265 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80168) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 14:53:35.857564 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 14:53:35.857843 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 14:53:35.858124 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 14:53:35.858400 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 14:53:35.858677 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 14:53:35.858946 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 14:53:35.874571 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80167) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 14:53:35.877557 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80168) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 14:53:35.877943 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80167) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 14:53:35.881060 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80168) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 14:53:35.881514 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 14:53:35.883128 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 14:53:35.884464 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 14:53:35.884750 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80167) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 14:53:35.885027 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 14:53:35.885299 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 14:53:35.886109 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 14:53:35.888290 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80168) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 14:53:35.888569 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 14:53:35.888854 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 14:53:35.893268 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 14:53:35.894893 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80167) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 14:53:35.896252 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 14:53:35.897939 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80168) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 14:53:35.898408 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 14:53:35.898699 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 14:53:35.898978 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 14:53:35.899246 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 14:53:35.901246 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 14:53:35.901600 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 14:53:35.901939 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 14:53:35.902263 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 14:53:35.908383 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80167) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 14:53:35.908662 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 14:53:35.909040 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 14:53:35.911365 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80168) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 14:53:35.911645 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 14:53:35.912021 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 14:53:35.916924 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80167) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 14:53:35.917198 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 14:53:35.917489 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 14:53:35.917759 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 14:53:35.918021 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 14:53:35.918283 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 14:53:35.918566 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 14:53:35.919350 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 14:53:35.919625 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 14:53:35.919893 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 14:53:35.920154 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 14:53:35.920407 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 14:53:35.920668 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 14:53:35.920747 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80168) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 26 14:53:35.920925 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80167) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 14:53:35.921060 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 14:53:35.921210 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 14:53:35.921496 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 14:53:35.921765 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 14:53:35.922031 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 14:53:35.922295 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 14:53:35.922569 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 14:53:35.922832 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 14:53:35.923109 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 14:53:35.923384 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 14:53:35.923638 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 14:53:35.923894 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 14:53:35.924152 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80168) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 26 14:53:35.960038 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55e9034ef9c0 pid: 80167 (default app) May 26 14:53:35.963274 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d467e225-9db2-4ef5-b5c7-b5164e495b61 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:53:35.963369 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55e9034ef9c0 pid: 80168 (default app) May 26 14:53:35.963563 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d467e225-9db2-4ef5-b5c7-b5164e495b61 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:53:35.963837 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d467e225-9db2-4ef5-b5c7-b5164e495b61 None None] PATH_INFO: `/v3` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:53:35.968150 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1/1] 10.222.0.46 () {50 vars in 791 bytes} [Thu May 26 14:53:35 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:53:44.050568 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06e67d5-3ad2-4a20-9601-a7e6bafc7a31 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:53:44.051183 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06e67d5-3ad2-4a20-9601-a7e6bafc7a31 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:53:44.051183 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06e67d5-3ad2-4a20-9601-a7e6bafc7a31 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:53:44.055886 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1/2] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 14:53:44 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 14:53:44.062929 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71754b3a-4082-4b96-bbcb-628f24c6fc8c None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:53:44.063456 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71754b3a-4082-4b96-bbcb-628f24c6fc8c None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:53:44.063635 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71754b3a-4082-4b96-bbcb-628f24c6fc8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:53:44.135894 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: INFO dbcounter [None req-71754b3a-4082-4b96-bbcb-628f24c6fc8c None None] Registered counter for database keystone May 26 14:53:44.159134 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG oslo_db.sqlalchemy.engines [None req-71754b3a-4082-4b96-bbcb-628f24c6fc8c None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80167) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 26 14:53:44.166260 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writer thread running {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 26 14:53:44.361688 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG passlib.handlers.bcrypt [None req-71754b3a-4082-4b96-bbcb-628f24c6fc8c None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80167) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} May 26 14:53:44.361951 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG passlib.handlers.bcrypt [None req-71754b3a-4082-4b96-bbcb-628f24c6fc8c None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80167) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} May 26 14:53:44.422546 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-71754b3a-4082-4b96-bbcb-628f24c6fc8c None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:53:44.424630 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-71754b3a-4082-4b96-bbcb-628f24c6fc8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:53:44.556519 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 2/3] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 14:53:44 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 495 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) May 26 14:53:44.710824 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06e67d5-3ad2-4a20-9601-a7e6bafc7a31 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:53:44.779767 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: INFO dbcounter [None req-f06e67d5-3ad2-4a20-9601-a7e6bafc7a31 None None] Registered counter for database keystone May 26 14:53:44.802154 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG oslo_db.sqlalchemy.engines [None req-f06e67d5-3ad2-4a20-9601-a7e6bafc7a31 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80168) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 26 14:53:44.808469 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writer thread running {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 26 14:53:44.930118 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06e67d5-3ad2-4a20-9601-a7e6bafc7a31 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:53:44.934885 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc58243-5e49-42ca-b91d-dc5727432ac3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:53:44.935562 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc58243-5e49-42ca-b91d-dc5727432ac3 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:53:44.935850 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc58243-5e49-42ca-b91d-dc5727432ac3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:53:44.936130 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc58243-5e49-42ca-b91d-dc5727432ac3 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:53:44.944654 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc58243-5e49-42ca-b91d-dc5727432ac3 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:53:44.996289 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: WARNING keystone.common.rbac_enforcer.enforcer [None req-9bc58243-5e49-42ca-b91d-dc5727432ac3 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. May 26 14:53:44.997212 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc58243-5e49-42ca-b91d-dc5727432ac3 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:53:45.007608 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 2/4] 10.222.0.46 () {58 vars in 1324 bytes} [Thu May 26 14:53:44 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) May 26 14:53:45.013848 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71754b3a-4082-4b96-bbcb-628f24c6fc8c None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:53:45.028885 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71754b3a-4082-4b96-bbcb-628f24c6fc8c None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:53:45.033524 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ccf644-875f-4708-a0ff-2338a3194eff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:53:45.034646 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ccf644-875f-4708-a0ff-2338a3194eff admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:53:45.034929 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ccf644-875f-4708-a0ff-2338a3194eff admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:53:45.035205 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ccf644-875f-4708-a0ff-2338a3194eff admin admin] PATH_INFO: `/v3/projects/064fc3c7af494c988e44e426f21b1c78` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:53:45.036732 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ccf644-875f-4708-a0ff-2338a3194eff admin admin] RBAC: Authorizing `identity:get_project(project_id=064fc3c7af494c988e44e426f21b1c78)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:53:45.196637 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: WARNING keystone.common.rbac_enforcer.enforcer [None req-26ccf644-875f-4708-a0ff-2338a3194eff admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. May 26 14:53:45.197259 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ccf644-875f-4708-a0ff-2338a3194eff admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:53:45.201697 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 3/5] 10.222.0.46 () {56 vars in 1207 bytes} [Thu May 26 14:53:45 2022] GET /identity/v3/projects/064fc3c7af494c988e44e426f21b1c78 => generated 344 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:53:47.365727 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71c1d2f-a323-426c-bcd4-3e07e2ea40e8 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:53:47.366279 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71c1d2f-a323-426c-bcd4-3e07e2ea40e8 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:53:47.366279 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71c1d2f-a323-426c-bcd4-3e07e2ea40e8 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:53:47.370496 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 3/6] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 14:53:47 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 14:53:47.377930 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677bf4ae-d2bf-4674-aabc-d3710cd33ddc None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:53:47.378444 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677bf4ae-d2bf-4674-aabc-d3710cd33ddc None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:53:47.378444 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677bf4ae-d2bf-4674-aabc-d3710cd33ddc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:53:47.400912 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-677bf4ae-d2bf-4674-aabc-d3710cd33ddc None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:53:47.402495 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-677bf4ae-d2bf-4674-aabc-d3710cd33ddc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:53:47.410703 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 4/7] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 14:53:47 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) May 26 14:53:47.433584 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71c1d2f-a323-426c-bcd4-3e07e2ea40e8 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:53:47.446371 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71c1d2f-a323-426c-bcd4-3e07e2ea40e8 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:53:47.450790 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2b93dd-2f67-4f22-9efe-95a264aec0a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:53:47.451439 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2b93dd-2f67-4f22-9efe-95a264aec0a5 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:53:47.451724 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2b93dd-2f67-4f22-9efe-95a264aec0a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:53:47.452009 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2b93dd-2f67-4f22-9efe-95a264aec0a5 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:53:47.460342 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2b93dd-2f67-4f22-9efe-95a264aec0a5 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:53:47.462249 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2b93dd-2f67-4f22-9efe-95a264aec0a5 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:53:47.472244 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 4/8] 10.222.0.46 () {58 vars in 1324 bytes} [Thu May 26 14:53:47 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 26 14:53:47.478559 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677bf4ae-d2bf-4674-aabc-d3710cd33ddc None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:53:47.488537 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677bf4ae-d2bf-4674-aabc-d3710cd33ddc None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:53:47.493013 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563592f2-336b-4c94-987f-08bd78f78ea6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:53:47.493931 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563592f2-336b-4c94-987f-08bd78f78ea6 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:53:47.494216 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563592f2-336b-4c94-987f-08bd78f78ea6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:53:47.494492 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563592f2-336b-4c94-987f-08bd78f78ea6 admin admin] PATH_INFO: `/v3/users/dede0ef3af5b453796bce68732b5fd29` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:53:47.495884 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563592f2-336b-4c94-987f-08bd78f78ea6 admin admin] RBAC: Authorizing `identity:get_user(user_id=dede0ef3af5b453796bce68732b5fd29)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:53:47.498426 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-563592f2-336b-4c94-987f-08bd78f78ea6 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:53:47.500243 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 5/9] 10.222.0.46 () {56 vars in 1198 bytes} [Thu May 26 14:53:47 2022] GET /identity/v3/users/dede0ef3af5b453796bce68732b5fd29 => generated 247 bytes in 23 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:53:49.847019 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8b69fd-4fba-42b3-a4ad-a16a1b1f7aaa None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:53:49.847533 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8b69fd-4fba-42b3-a4ad-a16a1b1f7aaa None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:53:49.847645 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8b69fd-4fba-42b3-a4ad-a16a1b1f7aaa None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:53:49.848908 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 5/10] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 14:53:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 14:53:49.854289 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e255b4c-36d4-4cae-8c92-1bcc560130d9 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:53:49.854724 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e255b4c-36d4-4cae-8c92-1bcc560130d9 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:53:49.854854 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e255b4c-36d4-4cae-8c92-1bcc560130d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:53:49.872242 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5916edd-374e-401c-99df-95dab2bdc019 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:53:49.872447 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-6e255b4c-36d4-4cae-8c92-1bcc560130d9 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:53:49.872544 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5916edd-374e-401c-99df-95dab2bdc019 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:53:49.872801 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5916edd-374e-401c-99df-95dab2bdc019 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:53:49.873645 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-6e255b4c-36d4-4cae-8c92-1bcc560130d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:53:49.874195 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 6/11] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 14:53:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 14:53:49.879628 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bd6a10-4e47-470b-8209-cb9fe6c0ea3d None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:53:49.879850 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bd6a10-4e47-470b-8209-cb9fe6c0ea3d None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:53:49.880137 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bd6a10-4e47-470b-8209-cb9fe6c0ea3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:53:49.921646 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG passlib.handlers.bcrypt [None req-75bd6a10-4e47-470b-8209-cb9fe6c0ea3d None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80168) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} May 26 14:53:49.922078 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG passlib.handlers.bcrypt [None req-75bd6a10-4e47-470b-8209-cb9fe6c0ea3d None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80168) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} May 26 14:53:49.958868 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 6/12] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 14:53:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 14:53:49.968892 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-75bd6a10-4e47-470b-8209-cb9fe6c0ea3d None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:53:49.970535 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-75bd6a10-4e47-470b-8209-cb9fe6c0ea3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:53:49.981982 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e255b4c-36d4-4cae-8c92-1bcc560130d9 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:53:50.006959 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:53:50.077580 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 7/13] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 14:53:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 14:53:50.095852 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e255b4c-36d4-4cae-8c92-1bcc560130d9 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:53:50.102133 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bd6a10-4e47-470b-8209-cb9fe6c0ea3d None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:53:50.132190 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d26f6c3-5a68-46ea-9e3e-2b142bb1cbe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:53:50.132774 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d26f6c3-5a68-46ea-9e3e-2b142bb1cbe9 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:53:50.133058 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d26f6c3-5a68-46ea-9e3e-2b142bb1cbe9 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:53:50.133341 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d26f6c3-5a68-46ea-9e3e-2b142bb1cbe9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:53:50.206225 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d26f6c3-5a68-46ea-9e3e-2b142bb1cbe9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:53:50.207725 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d26f6c3-5a68-46ea-9e3e-2b142bb1cbe9 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:53:50.232634 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bd6a10-4e47-470b-8209-cb9fe6c0ea3d None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:53:50.233701 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:53:50.235142 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:53:50.273760 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8529f7be-eba0-4914-a661-13f49466f1ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:53:50.274798 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8529f7be-eba0-4914-a661-13f49466f1ab None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:53:50.275085 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8529f7be-eba0-4914-a661-13f49466f1ab None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:53:50.275364 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8529f7be-eba0-4914-a661-13f49466f1ab None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:53:50.288387 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8529f7be-eba0-4914-a661-13f49466f1ab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 14:53:50.299848 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-8529f7be-eba0-4914-a661-13f49466f1ab None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 14:53:50.299848 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:53:50.299848 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:53:50.299848 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:53:50.299848 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:53:50.299848 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:53:50.299848 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:53:50.299848 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:53:50.299848 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:53:50.299848 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:53:50.299848 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:53:50.299848 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:53:50.299848 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 14:53:50.299848 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 14:53:50.299848 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 14:53:50.299848 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 14:53:50.300846 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 14:53:50.300846 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 14:53:50.300846 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 14:53:50.300846 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 14:53:50.300846 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:53:50.300846 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:53:50.300846 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:53:50.300846 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:53:50.300846 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:53:50.300846 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:53:50.300846 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:53:50.300846 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 26 14:53:50.300846 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:53:50.300846 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 26 14:53:50.300846 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:53:50.300846 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:53:50.301699 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:53:50.301699 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 26 14:53:50.301699 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:53:50.301699 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 26 14:53:50.301699 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 14:53:50.301699 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 14:53:50.301699 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 14:53:50.301699 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 14:53:50.301699 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 14:53:50.301699 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 14:53:50.301699 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 14:53:50.301699 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 26 14:53:50.301699 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 8/14] 10.222.0.46 () {56 vars in 1096 bytes} [Thu May 26 14:53:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:53:50.307208 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8529f7be-eba0-4914-a661-13f49466f1ab None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:53:50.315058 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 7/15] 10.222.0.46 () {58 vars in 1282 bytes} [Thu May 26 14:53:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 26 14:53:50.320856 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d26f6c3-5a68-46ea-9e3e-2b142bb1cbe9 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:53:50.416635 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8529f7be-eba0-4914-a661-13f49466f1ab None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:53:50.429718 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d26f6c3-5a68-46ea-9e3e-2b142bb1cbe9 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:53:50.457380 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:53:50.467520 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522824e2-c9e9-430b-931e-0b884c5ce83e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:53:50.468210 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522824e2-c9e9-430b-931e-0b884c5ce83e None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:53:50.468499 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522824e2-c9e9-430b-931e-0b884c5ce83e None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:53:50.468776 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522824e2-c9e9-430b-931e-0b884c5ce83e None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:53:50.469579 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522824e2-c9e9-430b-931e-0b884c5ce83e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:53:50.469891 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522824e2-c9e9-430b-931e-0b884c5ce83e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:53:50.470102 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f109d03c-1810-4a1e-92c8-dc59cbfbe654 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:53:50.470903 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f109d03c-1810-4a1e-92c8-dc59cbfbe654 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:53:50.471181 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f109d03c-1810-4a1e-92c8-dc59cbfbe654 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:53:50.471351 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-522824e2-c9e9-430b-931e-0b884c5ce83e None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:53:50.471436 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f109d03c-1810-4a1e-92c8-dc59cbfbe654 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:53:50.482454 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f109d03c-1810-4a1e-92c8-dc59cbfbe654 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:53:50.483977 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f109d03c-1810-4a1e-92c8-dc59cbfbe654 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:53:50.484891 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 9/16] 10.222.0.46 () {56 vars in 1099 bytes} [Thu May 26 14:53:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:53:50.491586 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522824e2-c9e9-430b-931e-0b884c5ce83e None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:53:50.508648 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-f109d03c-1810-4a1e-92c8-dc59cbfbe654 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 14:53:50.508648 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:53:50.508648 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:53:50.508648 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:53:50.508648 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:53:50.508648 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:53:50.508648 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:53:50.508648 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:53:50.508648 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:53:50.508648 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:53:50.508648 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:53:50.508648 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:53:50.508648 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 14:53:50.508648 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 14:53:50.508648 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 14:53:50.509633 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 14:53:50.509633 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:53:50.509633 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:53:50.509633 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:53:50.509633 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:53:50.509633 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:53:50.509633 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:53:50.509633 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:53:50.509633 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 26 14:53:50.509633 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:53:50.509633 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 26 14:53:50.509633 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:53:50.509633 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:53:50.509633 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:53:50.509633 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 26 14:53:50.509633 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:53:50.509633 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 26 14:53:50.510572 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 14:53:50.510572 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 14:53:50.510572 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 14:53:50.510572 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 26 14:53:50.510572 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 8/17] 10.222.0.46 () {56 vars in 1108 bytes} [Thu May 26 14:53:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 14:53:50.516654 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f109d03c-1810-4a1e-92c8-dc59cbfbe654 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:53:50.520840 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:53:50.611958 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-522824e2-c9e9-430b-931e-0b884c5ce83e None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:53:50.626496 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f109d03c-1810-4a1e-92c8-dc59cbfbe654 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:53:50.651239 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d51d81-d9a4-4101-bb07-e860780ccc68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:53:50.652131 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d51d81-d9a4-4101-bb07-e860780ccc68 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:53:50.652425 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d51d81-d9a4-4101-bb07-e860780ccc68 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:53:50.652729 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d51d81-d9a4-4101-bb07-e860780ccc68 None admin] PATH_INFO: `/v3/users/dede0ef3af5b453796bce68732b5fd29` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:53:50.654418 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d51d81-d9a4-4101-bb07-e860780ccc68 None admin] RBAC: Authorizing `identity:get_user(user_id=dede0ef3af5b453796bce68732b5fd29)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:53:50.656107 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d51d81-d9a4-4101-bb07-e860780ccc68 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:53:50.657939 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 10/18] 10.222.0.46 () {56 vars in 1177 bytes} [Thu May 26 14:53:50 2022] GET /identity/v3/users/dede0ef3af5b453796bce68732b5fd29 => generated 247 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 14:53:50.663081 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669e8c0a-d787-401e-836f-73235471715a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:53:50.663288 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d51d81-d9a4-4101-bb07-e860780ccc68 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:53:50.663752 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669e8c0a-d787-401e-836f-73235471715a None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:53:50.664038 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669e8c0a-d787-401e-836f-73235471715a None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:53:50.664319 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-669e8c0a-d787-401e-836f-73235471715a None admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:53:50.665054 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669e8c0a-d787-401e-836f-73235471715a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:53:50.665363 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669e8c0a-d787-401e-836f-73235471715a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:53:50.666863 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-669e8c0a-d787-401e-836f-73235471715a None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:53:50.685080 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 9/19] 10.222.0.46 () {56 vars in 1109 bytes} [Thu May 26 14:53:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 14:53:50.770816 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:53:50.778358 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d51d81-d9a4-4101-bb07-e860780ccc68 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:53:50.815083 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85234b3c-9551-4254-9271-e9fa2f968cca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:53:50.815899 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85234b3c-9551-4254-9271-e9fa2f968cca None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:53:50.816181 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85234b3c-9551-4254-9271-e9fa2f968cca None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:53:50.816458 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85234b3c-9551-4254-9271-e9fa2f968cca None admin] PATH_INFO: `/v3/domains/default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:53:50.817772 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85234b3c-9551-4254-9271-e9fa2f968cca None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:53:50.819285 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85234b3c-9551-4254-9271-e9fa2f968cca None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:53:50.821013 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 11/20] 10.222.0.46 () {56 vars in 1108 bytes} [Thu May 26 14:53:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 14:53:50.828100 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669e8c0a-d787-401e-836f-73235471715a None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:53:50.867885 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:53:50.953832 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-669e8c0a-d787-401e-836f-73235471715a None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:53:50.995429 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090a6a8d-3ef9-4dfb-be03-8a6dfc7243cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:53:50.996111 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090a6a8d-3ef9-4dfb-be03-8a6dfc7243cb None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:53:50.997009 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090a6a8d-3ef9-4dfb-be03-8a6dfc7243cb None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:53:50.997291 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090a6a8d-3ef9-4dfb-be03-8a6dfc7243cb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:53:50.998271 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090a6a8d-3ef9-4dfb-be03-8a6dfc7243cb None admin] RBAC: Adding query filter params (role.id=a2e551773cbd4df084020676865b8222, scope.domain.id=default, user.id=dede0ef3af5b453796bce68732b5fd29) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:53:50.998584 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090a6a8d-3ef9-4dfb-be03-8a6dfc7243cb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:53:51.000057 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090a6a8d-3ef9-4dfb-be03-8a6dfc7243cb None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:53:51.008127 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 10/21] 10.222.0.46 () {56 vars in 1322 bytes} [Thu May 26 14:53:50 2022] GET /identity/v3/role_assignments?user.id=dede0ef3af5b453796bce68732b5fd29&scope.domain.id=default&role.id=a2e551773cbd4df084020676865b8222 => generated 233 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:53:53.467792 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5acfb3-8e9a-464f-9eb2-d9e573f5ca19 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:53:53.468591 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5acfb3-8e9a-464f-9eb2-d9e573f5ca19 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:53:53.468716 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5acfb3-8e9a-464f-9eb2-d9e573f5ca19 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:53:53.471000 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 12/22] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 14:53:53 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 14:53:53.478184 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eea355a-5a75-409e-8d10-e21a93e885d1 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:53:53.479672 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eea355a-5a75-409e-8d10-e21a93e885d1 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:53:53.481157 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eea355a-5a75-409e-8d10-e21a93e885d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:53:53.521348 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-3eea355a-5a75-409e-8d10-e21a93e885d1 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:53:53.523785 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-3eea355a-5a75-409e-8d10-e21a93e885d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:53:53.637705 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 11/23] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 14:53:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 14:53:53.686933 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5acfb3-8e9a-464f-9eb2-d9e573f5ca19 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:53:53.839348 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5acfb3-8e9a-464f-9eb2-d9e573f5ca19 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:53:53.894727 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7be7050-9c15-449f-99af-6cdc47de376e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:53:53.895534 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7be7050-9c15-449f-99af-6cdc47de376e None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:53:53.895822 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7be7050-9c15-449f-99af-6cdc47de376e None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:53:53.896095 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7be7050-9c15-449f-99af-6cdc47de376e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:53:53.928644 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7be7050-9c15-449f-99af-6cdc47de376e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:53:53.930453 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7be7050-9c15-449f-99af-6cdc47de376e None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:53:54.006522 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cb5eb9-33ad-4c11-8e83-36df0ff88d86 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:53:54.006861 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cb5eb9-33ad-4c11-8e83-36df0ff88d86 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:53:54.007095 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cb5eb9-33ad-4c11-8e83-36df0ff88d86 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:53:54.008795 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 12/24] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 14:53:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 14:53:54.027958 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c9a9f9-8473-4132-b17a-e3a7dab7fef8 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:53:54.028303 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c9a9f9-8473-4132-b17a-e3a7dab7fef8 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:53:54.028524 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c9a9f9-8473-4132-b17a-e3a7dab7fef8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:53:54.066738 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-d8c9a9f9-8473-4132-b17a-e3a7dab7fef8 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:53:54.067938 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-d8c9a9f9-8473-4132-b17a-e3a7dab7fef8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:53:54.118520 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:53:54.133075 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-c7be7050-9c15-449f-99af-6cdc47de376e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 14:53:54.133075 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:53:54.133075 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:53:54.133075 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:53:54.133075 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:53:54.133075 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:53:54.133075 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:53:54.133075 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:53:54.133075 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:53:54.133075 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:53:54.133075 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:53:54.133075 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:53:54.133075 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 14:53:54.133075 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 14:53:54.133075 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 14:53:54.133991 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 14:53:54.133991 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:53:54.133991 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:53:54.133991 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:53:54.133991 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:53:54.133991 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:53:54.133991 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:53:54.133991 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:53:54.133991 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 26 14:53:54.133991 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:53:54.133991 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 26 14:53:54.133991 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:53:54.133991 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:53:54.133991 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:53:54.133991 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 26 14:53:54.133991 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:53:54.133991 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 26 14:53:54.134773 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 14:53:54.134773 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 14:53:54.134773 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 14:53:54.134773 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 26 14:53:54.143039 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 13/25] 10.222.0.46 () {56 vars in 1108 bytes} [Thu May 26 14:53:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 457 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:53:54.148804 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7be7050-9c15-449f-99af-6cdc47de376e None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:53:54.172924 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:53:54.253402 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 13/26] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 14:53:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 14:53:54.287942 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c9a9f9-8473-4132-b17a-e3a7dab7fef8 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:53:54.321067 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7be7050-9c15-449f-99af-6cdc47de376e None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:53:54.402325 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5037223-aaeb-410e-a194-7c862ab4790f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:53:54.403401 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5037223-aaeb-410e-a194-7c862ab4790f None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:53:54.403896 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5037223-aaeb-410e-a194-7c862ab4790f None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:53:54.404380 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5037223-aaeb-410e-a194-7c862ab4790f None admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:53:54.405365 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5037223-aaeb-410e-a194-7c862ab4790f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:53:54.405933 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5037223-aaeb-410e-a194-7c862ab4790f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:53:54.407610 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5037223-aaeb-410e-a194-7c862ab4790f None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:53:54.441485 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 14/27] 10.222.0.46 () {56 vars in 1109 bytes} [Thu May 26 14:53:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:53:54.442928 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5037223-aaeb-410e-a194-7c862ab4790f None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:53:54.543657 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:53:54.545657 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c9a9f9-8473-4132-b17a-e3a7dab7fef8 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:53:54.573278 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5037223-aaeb-410e-a194-7c862ab4790f None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:53:54.626371 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441c05d1-0e51-4a4a-8f3e-3b96be3603b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:53:54.627482 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441c05d1-0e51-4a4a-8f3e-3b96be3603b0 None admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:53:54.627926 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441c05d1-0e51-4a4a-8f3e-3b96be3603b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:53:54.628343 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441c05d1-0e51-4a4a-8f3e-3b96be3603b0 None admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:53:54.631858 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441c05d1-0e51-4a4a-8f3e-3b96be3603b0 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:53:54.634469 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441c05d1-0e51-4a4a-8f3e-3b96be3603b0 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:53:54.660039 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38f4d1e-5856-4096-8cc5-28fd5d68b4fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:53:54.661803 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38f4d1e-5856-4096-8cc5-28fd5d68b4fb None admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:53:54.662301 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38f4d1e-5856-4096-8cc5-28fd5d68b4fb None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:53:54.662780 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38f4d1e-5856-4096-8cc5-28fd5d68b4fb None admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:53:54.664067 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38f4d1e-5856-4096-8cc5-28fd5d68b4fb None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:53:54.665869 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38f4d1e-5856-4096-8cc5-28fd5d68b4fb None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:53:54.862460 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 14/28] 10.222.0.46 () {60 vars in 1131 bytes} [Thu May 26 14:53:54 2022] POST /identity/v3/roles => generated 219 bytes in 575 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 14:53:54.864515 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5549c442-0369-4ca1-a1d2-c18fd801b7d0 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:53:54.865038 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5549c442-0369-4ca1-a1d2-c18fd801b7d0 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:53:54.865600 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5549c442-0369-4ca1-a1d2-c18fd801b7d0 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:53:54.867989 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 15/29] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 14:53:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 14:53:54.875694 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf248ff-4fc4-48d3-a5e4-1c5251cc7212 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:53:54.876305 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf248ff-4fc4-48d3-a5e4-1c5251cc7212 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:53:54.876800 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf248ff-4fc4-48d3-a5e4-1c5251cc7212 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:53:54.894856 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 15/30] 10.222.0.46 () {60 vars in 1141 bytes} [Thu May 26 14:53:54 2022] POST /identity/v3/projects => generated 301 bytes in 452 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 14:53:54.898993 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-faf248ff-4fc4-48d3-a5e4-1c5251cc7212 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:53:54.901163 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-faf248ff-4fc4-48d3-a5e4-1c5251cc7212 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:53:54.918746 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:53:54.984030 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 16/31] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 14:53:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 14:53:55.020161 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441c05d1-0e51-4a4a-8f3e-3b96be3603b0 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:53:55.189278 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441c05d1-0e51-4a4a-8f3e-3b96be3603b0 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:53:55.238245 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:53:55.252717 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95abe84-7ecf-48ba-89d3-5c1118afc517 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:53:55.253640 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95abe84-7ecf-48ba-89d3-5c1118afc517 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:53:55.253938 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95abe84-7ecf-48ba-89d3-5c1118afc517 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:53:55.254227 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95abe84-7ecf-48ba-89d3-5c1118afc517 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:53:55.274412 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95abe84-7ecf-48ba-89d3-5c1118afc517 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 14:53:55.282546 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-d95abe84-7ecf-48ba-89d3-5c1118afc517 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 14:53:55.282546 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:53:55.282546 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:53:55.282546 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:53:55.282546 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:53:55.282546 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:53:55.282546 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:53:55.282546 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:53:55.282546 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:53:55.282546 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:53:55.282546 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:53:55.282546 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:53:55.282546 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 14:53:55.282546 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 14:53:55.282546 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 14:53:55.282546 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 14:53:55.283866 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 14:53:55.283866 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 14:53:55.283866 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 14:53:55.283866 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 14:53:55.283866 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:53:55.283866 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:53:55.283866 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:53:55.283866 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:53:55.283866 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:53:55.283866 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:53:55.283866 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:53:55.283866 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 26 14:53:55.283866 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:53:55.283866 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 26 14:53:55.283866 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:53:55.283866 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:53:55.285001 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:53:55.285001 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 26 14:53:55.285001 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:53:55.285001 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 26 14:53:55.285001 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 14:53:55.285001 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 14:53:55.285001 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 14:53:55.285001 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 14:53:55.285001 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 14:53:55.285001 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 14:53:55.285001 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 14:53:55.285001 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 26 14:53:55.285001 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 16/32] 10.222.0.46 () {56 vars in 1096 bytes} [Thu May 26 14:53:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:53:55.293517 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf248ff-4fc4-48d3-a5e4-1c5251cc7212 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:53:55.337031 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484b04e3-8591-442b-a696-9ec04e30f620 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:53:55.337580 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484b04e3-8591-442b-a696-9ec04e30f620 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:53:55.338009 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484b04e3-8591-442b-a696-9ec04e30f620 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:53:55.339556 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 17/33] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 14:53:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 14:53:55.346619 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd416c83-e145-4454-a61a-9c9f195ce628 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:53:55.346914 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd416c83-e145-4454-a61a-9c9f195ce628 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:53:55.347197 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd416c83-e145-4454-a61a-9c9f195ce628 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:53:55.377222 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-cd416c83-e145-4454-a61a-9c9f195ce628 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:53:55.379257 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-cd416c83-e145-4454-a61a-9c9f195ce628 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:53:55.437690 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf248ff-4fc4-48d3-a5e4-1c5251cc7212 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:53:55.469125 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 18/34] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 14:53:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 14:53:55.480209 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7fa47e-70d3-4e7f-8180-bed4d24824db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:53:55.481900 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7fa47e-70d3-4e7f-8180-bed4d24824db None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:53:55.482449 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7fa47e-70d3-4e7f-8180-bed4d24824db None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:53:55.483025 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7fa47e-70d3-4e7f-8180-bed4d24824db None admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:53:55.484124 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7fa47e-70d3-4e7f-8180-bed4d24824db None admin] RBAC: Adding query filter params (name=admin) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:53:55.484780 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7fa47e-70d3-4e7f-8180-bed4d24824db None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:53:55.486914 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7fa47e-70d3-4e7f-8180-bed4d24824db None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:53:55.493066 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd416c83-e145-4454-a61a-9c9f195ce628 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:53:55.506373 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 17/35] 10.222.0.46 () {56 vars in 1099 bytes} [Thu May 26 14:53:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:53:55.514215 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7fa47e-70d3-4e7f-8180-bed4d24824db None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:53:55.542006 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:53:55.637782 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd416c83-e145-4454-a61a-9c9f195ce628 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:53:55.655063 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:53:55.676696 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652a7195-ac69-48b4-bddd-26a85ec57a87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:53:55.677395 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652a7195-ac69-48b4-bddd-26a85ec57a87 None admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:53:55.677733 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652a7195-ac69-48b4-bddd-26a85ec57a87 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:53:55.678029 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652a7195-ac69-48b4-bddd-26a85ec57a87 None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:53:55.678821 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652a7195-ac69-48b4-bddd-26a85ec57a87 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:53:55.680427 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652a7195-ac69-48b4-bddd-26a85ec57a87 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:53:55.692002 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7fa47e-70d3-4e7f-8180-bed4d24824db None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:53:55.700592 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 19/36] 10.222.0.46 () {60 vars in 1131 bytes} [Thu May 26 14:53:55 2022] POST /identity/v3/roles => generated 225 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 14:53:55.701795 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b0f849-634c-4931-b3c4-7b0f19d5df25 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:53:55.702415 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b0f849-634c-4931-b3c4-7b0f19d5df25 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:53:55.702415 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b0f849-634c-4931-b3c4-7b0f19d5df25 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:53:55.704172 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 20/37] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 14:53:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 14:53:55.710165 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8537cbb7-f141-4b5f-ba3f-ebb9f70f9e2a None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:53:55.710807 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8537cbb7-f141-4b5f-ba3f-ebb9f70f9e2a None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:53:55.711359 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8537cbb7-f141-4b5f-ba3f-ebb9f70f9e2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:53:55.738866 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-8537cbb7-f141-4b5f-ba3f-ebb9f70f9e2a None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:53:55.741704 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-8537cbb7-f141-4b5f-ba3f-ebb9f70f9e2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:53:55.744874 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0549ffd-eb0d-498e-91d9-e0b5edc4f578 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:53:55.745777 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0549ffd-eb0d-498e-91d9-e0b5edc4f578 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:53:55.746178 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0549ffd-eb0d-498e-91d9-e0b5edc4f578 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:53:55.747008 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0549ffd-eb0d-498e-91d9-e0b5edc4f578 None admin] PATH_INFO: `/v3/users/dede0ef3af5b453796bce68732b5fd29` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:53:55.749139 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0549ffd-eb0d-498e-91d9-e0b5edc4f578 None admin] RBAC: Authorizing `identity:get_user(user_id=dede0ef3af5b453796bce68732b5fd29)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:53:55.750730 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0549ffd-eb0d-498e-91d9-e0b5edc4f578 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:53:55.752720 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 18/38] 10.222.0.46 () {56 vars in 1177 bytes} [Thu May 26 14:53:55 2022] GET /identity/v3/users/dede0ef3af5b453796bce68732b5fd29 => generated 247 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 14:53:55.758698 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0549ffd-eb0d-498e-91d9-e0b5edc4f578 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:53:55.846234 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 21/39] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 14:53:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 14:53:55.873144 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:53:55.879699 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8537cbb7-f141-4b5f-ba3f-ebb9f70f9e2a None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:53:55.910555 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0549ffd-eb0d-498e-91d9-e0b5edc4f578 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:53:55.959514 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff4f9aa-9bdb-43f1-82c4-0ec7ae660924 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:53:55.960330 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff4f9aa-9bdb-43f1-82c4-0ec7ae660924 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:53:55.960624 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff4f9aa-9bdb-43f1-82c4-0ec7ae660924 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:53:55.960934 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff4f9aa-9bdb-43f1-82c4-0ec7ae660924 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:53:55.962410 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff4f9aa-9bdb-43f1-82c4-0ec7ae660924 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:53:55.963877 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff4f9aa-9bdb-43f1-82c4-0ec7ae660924 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:53:55.966076 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 19/40] 10.222.0.46 () {56 vars in 1108 bytes} [Thu May 26 14:53:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 14:53:55.967286 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2774bce7-6715-4ae9-a646-6f6513d99011 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:53:55.967581 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2774bce7-6715-4ae9-a646-6f6513d99011 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:53:55.967852 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2774bce7-6715-4ae9-a646-6f6513d99011 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:53:55.969252 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 20/41] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 14:53:55 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 14:53:55.971572 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2774bce7-6715-4ae9-a646-6f6513d99011 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:53:55.997408 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8537cbb7-f141-4b5f-ba3f-ebb9f70f9e2a None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:53:56.005137 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:53:56.034712 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8433daa-2f38-46d2-b803-1d4aeeca6926 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:53:56.035510 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8433daa-2f38-46d2-b803-1d4aeeca6926 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:53:56.035792 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8433daa-2f38-46d2-b803-1d4aeeca6926 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:53:56.036079 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8433daa-2f38-46d2-b803-1d4aeeca6926 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:53:56.037246 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8433daa-2f38-46d2-b803-1d4aeeca6926 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:53:56.038811 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8433daa-2f38-46d2-b803-1d4aeeca6926 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:53:56.040723 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 22/42] 10.222.0.46 () {56 vars in 1108 bytes} [Thu May 26 14:53:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 14:53:56.041898 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd48e5cd-ef5e-4cc0-be1a-db2f184fe7d9 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:53:56.042186 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd48e5cd-ef5e-4cc0-be1a-db2f184fe7d9 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:53:56.042460 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd48e5cd-ef5e-4cc0-be1a-db2f184fe7d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:53:56.059997 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-fd48e5cd-ef5e-4cc0-be1a-db2f184fe7d9 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:53:56.061204 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-fd48e5cd-ef5e-4cc0-be1a-db2f184fe7d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:53:56.079558 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2774bce7-6715-4ae9-a646-6f6513d99011 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:53:56.115554 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e37c3f-8b7a-483a-a709-9891f16b2276 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:53:56.116435 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e37c3f-8b7a-483a-a709-9891f16b2276 None admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:53:56.116726 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e37c3f-8b7a-483a-a709-9891f16b2276 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:53:56.117013 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e37c3f-8b7a-483a-a709-9891f16b2276 None admin] PATH_INFO: `/v3/domains/default/users/dede0ef3af5b453796bce68732b5fd29/roles/a2e551773cbd4df084020676865b8222` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:53:56.123267 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e37c3f-8b7a-483a-a709-9891f16b2276 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=dede0ef3af5b453796bce68732b5fd29, role_id=a2e551773cbd4df084020676865b8222)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:53:56.124746 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e37c3f-8b7a-483a-a709-9891f16b2276 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:53:56.130246 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 23/43] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 14:53:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 14:53:56.131437 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd48e5cd-ef5e-4cc0-be1a-db2f184fe7d9 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:53:56.156876 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 21/44] 10.222.0.46 () {58 vars in 1361 bytes} [Thu May 26 14:53:55 2022] PUT /identity/v3/domains/default/users/dede0ef3af5b453796bce68732b5fd29/roles/a2e551773cbd4df084020676865b8222 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 14:53:56.157710 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e37c3f-8b7a-483a-a709-9891f16b2276 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:53:56.178995 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:53:56.253941 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd48e5cd-ef5e-4cc0-be1a-db2f184fe7d9 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:53:56.256773 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:53:56.288393 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e37c3f-8b7a-483a-a709-9891f16b2276 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:53:56.296072 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6fb91f-1dd0-45b7-84cc-f65703ae089e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:53:56.296768 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6fb91f-1dd0-45b7-84cc-f65703ae089e None admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:53:56.297055 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6fb91f-1dd0-45b7-84cc-f65703ae089e None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:53:56.297331 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6fb91f-1dd0-45b7-84cc-f65703ae089e None admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:53:56.298092 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6fb91f-1dd0-45b7-84cc-f65703ae089e None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:53:56.299577 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6fb91f-1dd0-45b7-84cc-f65703ae089e None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:53:56.328796 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 24/45] 10.222.0.46 () {60 vars in 1141 bytes} [Thu May 26 14:53:56 2022] POST /identity/v3/projects => generated 312 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 14:53:56.339192 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba90e621-8bdf-466e-81fa-65ea55b46a26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:53:56.339924 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba90e621-8bdf-466e-81fa-65ea55b46a26 None admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:53:56.340214 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba90e621-8bdf-466e-81fa-65ea55b46a26 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:53:56.340494 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba90e621-8bdf-466e-81fa-65ea55b46a26 None admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:53:56.341277 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba90e621-8bdf-466e-81fa-65ea55b46a26 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:53:56.342779 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba90e621-8bdf-466e-81fa-65ea55b46a26 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:53:56.369378 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 22/46] 10.222.0.46 () {60 vars in 1131 bytes} [Thu May 26 14:53:56 2022] POST /identity/v3/roles => generated 223 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 14:53:58.409153 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a56f899-1302-4f4c-939b-67947fa1401f None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:53:58.409739 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a56f899-1302-4f4c-939b-67947fa1401f None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:53:58.409739 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a56f899-1302-4f4c-939b-67947fa1401f None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:53:58.411134 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 25/47] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 14:53:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 14:53:58.416651 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed08ac2-e108-4a8a-b4aa-62cb324f7076 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:53:58.417200 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed08ac2-e108-4a8a-b4aa-62cb324f7076 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:53:58.417200 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed08ac2-e108-4a8a-b4aa-62cb324f7076 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:53:58.440552 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-fed08ac2-e108-4a8a-b4aa-62cb324f7076 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:53:58.441878 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-fed08ac2-e108-4a8a-b4aa-62cb324f7076 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:53:58.508329 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:53:58.520296 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 23/48] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 14:53:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 14:53:58.543607 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a56f899-1302-4f4c-939b-67947fa1401f None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:53:58.624807 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31939353-6e6d-454b-87e3-7c8b2d5a63b3 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:53:58.625110 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31939353-6e6d-454b-87e3-7c8b2d5a63b3 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:53:58.625399 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31939353-6e6d-454b-87e3-7c8b2d5a63b3 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:53:58.626814 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 24/49] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 14:53:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 14:53:58.632417 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7737409-a512-4f1b-8510-fb459de92276 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:53:58.632759 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7737409-a512-4f1b-8510-fb459de92276 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:53:58.633101 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7737409-a512-4f1b-8510-fb459de92276 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:53:58.653668 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-d7737409-a512-4f1b-8510-fb459de92276 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:53:58.655014 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-d7737409-a512-4f1b-8510-fb459de92276 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:53:58.671550 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a56f899-1302-4f4c-939b-67947fa1401f None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:53:58.711189 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8864f3ac-1fa2-45ac-97d9-bf5de0e49161 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:53:58.711995 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8864f3ac-1fa2-45ac-97d9-bf5de0e49161 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:53:58.712284 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8864f3ac-1fa2-45ac-97d9-bf5de0e49161 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:53:58.712640 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8864f3ac-1fa2-45ac-97d9-bf5de0e49161 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:53:58.724486 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8864f3ac-1fa2-45ac-97d9-bf5de0e49161 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 14:53:58.732207 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-8864f3ac-1fa2-45ac-97d9-bf5de0e49161 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 14:53:58.732207 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:53:58.732207 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:53:58.732207 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:53:58.732207 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:53:58.732207 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:53:58.732207 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:53:58.732207 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:53:58.732207 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:53:58.732207 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:53:58.732207 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:53:58.732207 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:53:58.732207 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 14:53:58.732207 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 14:53:58.732207 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 14:53:58.732207 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 14:53:58.733067 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 14:53:58.733067 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 14:53:58.733067 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 14:53:58.733067 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 14:53:58.733067 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:53:58.733067 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:53:58.733067 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:53:58.733067 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:53:58.733067 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:53:58.733067 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:53:58.733067 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:53:58.733067 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 26 14:53:58.733067 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:53:58.733067 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 26 14:53:58.733067 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:53:58.733067 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:53:58.733925 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:53:58.733925 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 26 14:53:58.733925 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:53:58.733925 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 26 14:53:58.733925 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 14:53:58.733925 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 14:53:58.733925 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 14:53:58.733925 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 14:53:58.733925 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 14:53:58.733925 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 14:53:58.733925 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 14:53:58.733925 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 26 14:53:58.733925 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 26/50] 10.222.0.46 () {56 vars in 1096 bytes} [Thu May 26 14:53:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:53:58.738080 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 25/51] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 14:53:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 14:53:58.740075 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8864f3ac-1fa2-45ac-97d9-bf5de0e49161 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:53:58.762885 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7737409-a512-4f1b-8510-fb459de92276 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:53:58.767454 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:53:58.858341 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8864f3ac-1fa2-45ac-97d9-bf5de0e49161 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:53:58.872118 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7737409-a512-4f1b-8510-fb459de92276 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:53:58.896671 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c6dd70-b220-4858-83e5-43ae101b11d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:53:58.897357 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c6dd70-b220-4858-83e5-43ae101b11d1 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:53:58.897676 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c6dd70-b220-4858-83e5-43ae101b11d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:53:58.897964 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c6dd70-b220-4858-83e5-43ae101b11d1 None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:53:58.898728 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c6dd70-b220-4858-83e5-43ae101b11d1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:53:58.899045 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c6dd70-b220-4858-83e5-43ae101b11d1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:53:58.899259 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:53:58.900507 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c6dd70-b220-4858-83e5-43ae101b11d1 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:53:58.909626 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 27/52] 10.222.0.46 () {56 vars in 1099 bytes} [Thu May 26 14:53:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:53:58.910323 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806382b2-dc01-40cc-9c3f-f7e11ab14dbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:53:58.911128 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806382b2-dc01-40cc-9c3f-f7e11ab14dbf None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:53:58.911416 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806382b2-dc01-40cc-9c3f-f7e11ab14dbf None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:53:58.911697 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806382b2-dc01-40cc-9c3f-f7e11ab14dbf None admin] PATH_INFO: `/v3/domains/default` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:53:58.912928 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806382b2-dc01-40cc-9c3f-f7e11ab14dbf None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:53:58.914722 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806382b2-dc01-40cc-9c3f-f7e11ab14dbf None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:53:58.915618 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c6dd70-b220-4858-83e5-43ae101b11d1 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:53:58.916110 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 26/53] 10.222.0.46 () {56 vars in 1108 bytes} [Thu May 26 14:53:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 14:53:58.921904 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806382b2-dc01-40cc-9c3f-f7e11ab14dbf None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:53:59.001118 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:53:59.025413 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c6dd70-b220-4858-83e5-43ae101b11d1 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:53:59.029674 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806382b2-dc01-40cc-9c3f-f7e11ab14dbf None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:53:59.061283 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c68b482-3f50-48cd-85b5-2a4ce1e9d67e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:53:59.062200 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c68b482-3f50-48cd-85b5-2a4ce1e9d67e None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:53:59.062491 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c68b482-3f50-48cd-85b5-2a4ce1e9d67e None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:53:59.062774 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c68b482-3f50-48cd-85b5-2a4ce1e9d67e None admin] PATH_INFO: `/v3/users/dede0ef3af5b453796bce68732b5fd29` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:53:59.064029 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c68b482-3f50-48cd-85b5-2a4ce1e9d67e None admin] RBAC: Authorizing `identity:get_user(user_id=dede0ef3af5b453796bce68732b5fd29)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:53:59.066106 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c68b482-3f50-48cd-85b5-2a4ce1e9d67e None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:53:59.066798 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87263658-b3f3-497c-93b6-555217f5b529 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:53:59.067235 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 28/54] 10.222.0.46 () {56 vars in 1177 bytes} [Thu May 26 14:53:58 2022] GET /identity/v3/users/dede0ef3af5b453796bce68732b5fd29 => generated 247 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 14:53:59.067505 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87263658-b3f3-497c-93b6-555217f5b529 None admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:53:59.067760 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87263658-b3f3-497c-93b6-555217f5b529 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:53:59.068041 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87263658-b3f3-497c-93b6-555217f5b529 None admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:53:59.068758 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87263658-b3f3-497c-93b6-555217f5b529 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:53:59.070266 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87263658-b3f3-497c-93b6-555217f5b529 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:53:59.072788 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c68b482-3f50-48cd-85b5-2a4ce1e9d67e None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:53:59.099195 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 27/55] 10.222.0.46 () {60 vars in 1140 bytes} [Thu May 26 14:53:58 2022] POST /identity/v3/projects => generated 298 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 14:53:59.202504 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c68b482-3f50-48cd-85b5-2a4ce1e9d67e None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:53:59.239628 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85532c7b-3f8d-42d7-8f3b-0c7ad4adec12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:53:59.240447 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85532c7b-3f8d-42d7-8f3b-0c7ad4adec12 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:53:59.240735 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85532c7b-3f8d-42d7-8f3b-0c7ad4adec12 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:53:59.241016 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85532c7b-3f8d-42d7-8f3b-0c7ad4adec12 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:53:59.242359 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85532c7b-3f8d-42d7-8f3b-0c7ad4adec12 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:53:59.243860 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85532c7b-3f8d-42d7-8f3b-0c7ad4adec12 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:53:59.245569 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 29/56] 10.222.0.46 () {56 vars in 1108 bytes} [Thu May 26 14:53:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 14:53:59.252433 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87263658-b3f3-497c-93b6-555217f5b529 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:53:59.305107 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:53:59.379284 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87263658-b3f3-497c-93b6-555217f5b529 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:53:59.420000 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0301e91b-2eef-40ed-af87-a86c534cc640 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:53:59.420792 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0301e91b-2eef-40ed-af87-a86c534cc640 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:53:59.421089 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0301e91b-2eef-40ed-af87-a86c534cc640 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:53:59.421392 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0301e91b-2eef-40ed-af87-a86c534cc640 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:53:59.422307 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0301e91b-2eef-40ed-af87-a86c534cc640 None admin] RBAC: Adding query filter params (role.id=a2e551773cbd4df084020676865b8222, scope.domain.id=default, user.id=dede0ef3af5b453796bce68732b5fd29) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:53:59.422628 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0301e91b-2eef-40ed-af87-a86c534cc640 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:53:59.424125 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0301e91b-2eef-40ed-af87-a86c534cc640 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:53:59.430338 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 28/57] 10.222.0.46 () {56 vars in 1322 bytes} [Thu May 26 14:53:59 2022] GET /identity/v3/role_assignments?user.id=dede0ef3af5b453796bce68732b5fd29&scope.domain.id=default&role.id=a2e551773cbd4df084020676865b8222 => generated 530 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:54:01.385047 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aef3c64-c6d6-4b10-b5ea-c640ef047f8f None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:01.385638 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aef3c64-c6d6-4b10-b5ea-c640ef047f8f None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:01.385638 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aef3c64-c6d6-4b10-b5ea-c640ef047f8f None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:01.387631 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 30/58] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 14:54:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 14:54:01.392812 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eea3473-57f7-47da-97f8-bdbff37b4101 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:01.393227 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eea3473-57f7-47da-97f8-bdbff37b4101 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:01.393378 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eea3473-57f7-47da-97f8-bdbff37b4101 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:01.410007 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-9eea3473-57f7-47da-97f8-bdbff37b4101 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:54:01.411217 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-9eea3473-57f7-47da-97f8-bdbff37b4101 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:54:01.478010 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 29/59] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 14:54:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 14:54:01.503255 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aef3c64-c6d6-4b10-b5ea-c640ef047f8f None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:01.519747 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:01.613955 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aef3c64-c6d6-4b10-b5ea-c640ef047f8f None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:01.650166 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8869cf-b148-41fc-ad15-5766db687814 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:01.650920 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8869cf-b148-41fc-ad15-5766db687814 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:01.651203 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8869cf-b148-41fc-ad15-5766db687814 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:01.651503 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8869cf-b148-41fc-ad15-5766db687814 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:01.652781 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8869cf-b148-41fc-ad15-5766db687814 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:54:01.654331 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8869cf-b148-41fc-ad15-5766db687814 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:54:01.656020 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 31/60] 10.222.0.46 () {56 vars in 1108 bytes} [Thu May 26 14:54:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:54:01.661693 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eea3473-57f7-47da-97f8-bdbff37b4101 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:01.703922 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:01.760932 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eea3473-57f7-47da-97f8-bdbff37b4101 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:01.795641 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0041f39c-09bd-48ff-ad98-bb6c56e6bfdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:01.796306 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0041f39c-09bd-48ff-ad98-bb6c56e6bfdc None admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:01.796593 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0041f39c-09bd-48ff-ad98-bb6c56e6bfdc None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:01.796875 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0041f39c-09bd-48ff-ad98-bb6c56e6bfdc None admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:01.797609 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0041f39c-09bd-48ff-ad98-bb6c56e6bfdc None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:54:01.799080 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0041f39c-09bd-48ff-ad98-bb6c56e6bfdc None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:54:01.828106 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 30/61] 10.222.0.46 () {60 vars in 1132 bytes} [Thu May 26 14:54:01 2022] POST /identity/v3/users => generated 275 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 14:54:05.422958 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028ec180-9159-462e-b66a-fc7a3fa3581f None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:05.424040 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028ec180-9159-462e-b66a-fc7a3fa3581f None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:05.424601 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028ec180-9159-462e-b66a-fc7a3fa3581f None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:05.426719 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 32/62] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 14:54:05 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 14:54:05.433101 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d12479-a212-4d01-a86e-070a6ddd167c None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:05.434097 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d12479-a212-4d01-a86e-070a6ddd167c None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:05.435125 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d12479-a212-4d01-a86e-070a6ddd167c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:05.463820 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-c2d12479-a212-4d01-a86e-070a6ddd167c None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:54:05.467425 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-c2d12479-a212-4d01-a86e-070a6ddd167c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:54:05.560094 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 31/63] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 14:54:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 14:54:05.589630 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028ec180-9159-462e-b66a-fc7a3fa3581f None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:05.605376 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19b5be7-343e-4380-95e2-1df7cbd20174 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:05.606249 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19b5be7-343e-4380-95e2-1df7cbd20174 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:05.606776 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19b5be7-343e-4380-95e2-1df7cbd20174 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:05.608539 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 32/64] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 14:54:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 14:54:05.615164 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7398bf61-c383-4328-8aa6-e2c0fb6c461d None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:05.616037 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7398bf61-c383-4328-8aa6-e2c0fb6c461d None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:05.616708 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7398bf61-c383-4328-8aa6-e2c0fb6c461d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:05.647210 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-7398bf61-c383-4328-8aa6-e2c0fb6c461d None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:54:05.648612 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-7398bf61-c383-4328-8aa6-e2c0fb6c461d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:54:05.697909 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:05.761477 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 33/65] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 14:54:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 14:54:05.773848 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:05.786446 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028ec180-9159-462e-b66a-fc7a3fa3581f None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:05.805423 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7398bf61-c383-4328-8aa6-e2c0fb6c461d None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:05.844863 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d57e62a-bd27-471b-8d30-9ada17036faf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:05.845704 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d57e62a-bd27-471b-8d30-9ada17036faf None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:05.845994 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d57e62a-bd27-471b-8d30-9ada17036faf None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:05.846276 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d57e62a-bd27-471b-8d30-9ada17036faf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:05.858471 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d57e62a-bd27-471b-8d30-9ada17036faf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 14:54:05.867178 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-5d57e62a-bd27-471b-8d30-9ada17036faf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 14:54:05.867178 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:54:05.867178 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:54:05.867178 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:54:05.867178 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:54:05.867178 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:54:05.867178 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:54:05.867178 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:54:05.867178 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:54:05.867178 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:54:05.867178 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:54:05.867178 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:54:05.867178 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 14:54:05.867178 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 14:54:05.867178 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 14:54:05.867178 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 14:54:05.868073 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 14:54:05.868073 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 14:54:05.868073 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 14:54:05.868073 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 14:54:05.868073 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:54:05.868073 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:54:05.868073 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:54:05.868073 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:54:05.868073 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:54:05.868073 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:54:05.868073 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:54:05.868073 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 26 14:54:05.868073 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:54:05.868073 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 26 14:54:05.868073 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:54:05.868073 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:54:05.868834 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:54:05.868834 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 26 14:54:05.868834 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:54:05.868834 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 26 14:54:05.868834 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 14:54:05.868834 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 14:54:05.868834 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 14:54:05.868834 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 14:54:05.868834 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 14:54:05.868834 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 14:54:05.868834 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 14:54:05.868834 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 26 14:54:05.868834 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 33/66] 10.222.0.46 () {56 vars in 1096 bytes} [Thu May 26 14:54:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:54:05.870242 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93a5b2a-2c1a-492c-844d-731fe32eec41 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:05.870424 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93a5b2a-2c1a-492c-844d-731fe32eec41 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:05.870850 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93a5b2a-2c1a-492c-844d-731fe32eec41 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:05.877264 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 34/67] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 14:54:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 14:54:05.878387 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93a5b2a-2c1a-492c-844d-731fe32eec41 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:05.948187 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7398bf61-c383-4328-8aa6-e2c0fb6c461d None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:05.995139 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d78b1d-d5ea-41be-9490-49a83a6c9f06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:05.995954 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d78b1d-d5ea-41be-9490-49a83a6c9f06 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:05.996238 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d78b1d-d5ea-41be-9490-49a83a6c9f06 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:05.996518 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d78b1d-d5ea-41be-9490-49a83a6c9f06 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:06.003071 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93a5b2a-2c1a-492c-844d-731fe32eec41 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:06.010272 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d78b1d-d5ea-41be-9490-49a83a6c9f06 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 14:54:06.019486 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-d0d78b1d-d5ea-41be-9490-49a83a6c9f06 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 26 14:54:06.019486 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:54:06.019486 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:54:06.019486 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:54:06.019486 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:54:06.019486 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:54:06.019486 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:54:06.019486 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:54:06.019486 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:54:06.019486 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:54:06.019486 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:54:06.019486 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:54:06.019486 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 14:54:06.019486 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 14:54:06.019486 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 14:54:06.019486 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 14:54:06.020603 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 14:54:06.020603 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 14:54:06.020603 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 14:54:06.020603 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 14:54:06.020603 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:54:06.020603 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:54:06.020603 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:54:06.020603 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:54:06.020603 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:54:06.020603 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:54:06.020603 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:54:06.020603 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 26 14:54:06.020603 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:54:06.020603 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 26 14:54:06.020603 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:54:06.020603 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:54:06.026445 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:54:06.026445 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 26 14:54:06.026445 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:54:06.026445 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 26 14:54:06.026445 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 14:54:06.026445 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 14:54:06.026445 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 14:54:06.026445 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 14:54:06.026445 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 14:54:06.026445 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 14:54:06.026445 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 26 14:54:06.026445 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 26 14:54:06.026445 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 34/68] 10.222.0.46 () {56 vars in 1099 bytes} [Thu May 26 14:54:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 14:54:06.026445 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492f4ef4-b7a2-4f21-89df-a6f7d8dbfb20 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:06.027112 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492f4ef4-b7a2-4f21-89df-a6f7d8dbfb20 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:06.027112 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492f4ef4-b7a2-4f21-89df-a6f7d8dbfb20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:06.040713 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-492f4ef4-b7a2-4f21-89df-a6f7d8dbfb20 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:54:06.041974 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-492f4ef4-b7a2-4f21-89df-a6f7d8dbfb20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:54:06.056323 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:06.068750 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d422d1da-ca76-4a04-8576-24f45dceef46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:06.069772 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d422d1da-ca76-4a04-8576-24f45dceef46 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:06.069928 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d422d1da-ca76-4a04-8576-24f45dceef46 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:06.070228 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d422d1da-ca76-4a04-8576-24f45dceef46 None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:06.071274 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d422d1da-ca76-4a04-8576-24f45dceef46 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:54:06.071594 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d422d1da-ca76-4a04-8576-24f45dceef46 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:54:06.073030 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d422d1da-ca76-4a04-8576-24f45dceef46 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:54:06.082880 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 35/69] 10.222.0.46 () {56 vars in 1099 bytes} [Thu May 26 14:54:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:54:06.084254 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cb655e-a2a3-4d55-b67c-1541963b7130 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:06.084541 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cb655e-a2a3-4d55-b67c-1541963b7130 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:06.084838 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cb655e-a2a3-4d55-b67c-1541963b7130 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:06.086149 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 36/70] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 14:54:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 14:54:06.087432 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cb655e-a2a3-4d55-b67c-1541963b7130 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:06.094950 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:06.143025 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 35/71] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 14:54:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 14:54:06.144310 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492f4ef4-b7a2-4f21-89df-a6f7d8dbfb20 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:06.239336 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cb655e-a2a3-4d55-b67c-1541963b7130 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:06.281544 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88977f5e-2cd7-47c3-913c-94d39f7e7141 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:06.282314 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88977f5e-2cd7-47c3-913c-94d39f7e7141 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:06.282666 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88977f5e-2cd7-47c3-913c-94d39f7e7141 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:06.283005 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88977f5e-2cd7-47c3-913c-94d39f7e7141 None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:06.283926 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88977f5e-2cd7-47c3-913c-94d39f7e7141 None admin] RBAC: Adding query filter params (name=member) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:54:06.284313 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88977f5e-2cd7-47c3-913c-94d39f7e7141 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:54:06.286027 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88977f5e-2cd7-47c3-913c-94d39f7e7141 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:54:06.289136 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492f4ef4-b7a2-4f21-89df-a6f7d8dbfb20 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:06.296700 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 37/72] 10.222.0.46 () {56 vars in 1101 bytes} [Thu May 26 14:54:06 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:54:06.298964 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee247707-156b-4567-b6fe-b56182e1d86e None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:06.299262 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee247707-156b-4567-b6fe-b56182e1d86e None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:06.299554 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee247707-156b-4567-b6fe-b56182e1d86e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:06.320024 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-ee247707-156b-4567-b6fe-b56182e1d86e None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:54:06.321847 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-ee247707-156b-4567-b6fe-b56182e1d86e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:54:06.326618 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fe1851-392c-464d-aa5f-364f6b5003b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:06.327564 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fe1851-392c-464d-aa5f-364f6b5003b4 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:06.327870 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fe1851-392c-464d-aa5f-364f6b5003b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:06.328158 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53fe1851-392c-464d-aa5f-364f6b5003b4 None admin] PATH_INFO: `/v3/users/dede0ef3af5b453796bce68732b5fd29` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:06.329547 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53fe1851-392c-464d-aa5f-364f6b5003b4 None admin] RBAC: Authorizing `identity:get_user(user_id=dede0ef3af5b453796bce68732b5fd29)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:54:06.331094 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53fe1851-392c-464d-aa5f-364f6b5003b4 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:54:06.333056 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 36/73] 10.222.0.46 () {56 vars in 1177 bytes} [Thu May 26 14:54:06 2022] GET /identity/v3/users/dede0ef3af5b453796bce68732b5fd29 => generated 247 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:54:06.333688 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fe1851-392c-464d-aa5f-364f6b5003b4 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:06.350564 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:06.396005 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:06.404740 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 38/74] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 14:54:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 14:54:06.406235 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8523dd89-bc67-4844-9eee-c47bdc303eb4 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:06.406534 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8523dd89-bc67-4844-9eee-c47bdc303eb4 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:06.406821 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8523dd89-bc67-4844-9eee-c47bdc303eb4 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:06.408139 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 39/75] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 14:54:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 14:54:06.408859 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8523dd89-bc67-4844-9eee-c47bdc303eb4 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:06.454424 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53fe1851-392c-464d-aa5f-364f6b5003b4 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:06.498858 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c63ce9-34c6-49f4-a151-f7725fb70217 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:06.499626 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c63ce9-34c6-49f4-a151-f7725fb70217 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:06.499917 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c63ce9-34c6-49f4-a151-f7725fb70217 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:06.500239 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c63ce9-34c6-49f4-a151-f7725fb70217 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:06.514465 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c63ce9-34c6-49f4-a151-f7725fb70217 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 14:54:06.522782 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-d8c63ce9-34c6-49f4-a151-f7725fb70217 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 14:54:06.522782 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:54:06.522782 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:54:06.522782 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:54:06.522782 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:54:06.522782 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:54:06.522782 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:54:06.522782 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:54:06.522782 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:54:06.522782 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:54:06.522782 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:54:06.522782 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:54:06.522782 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 14:54:06.522782 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 14:54:06.522782 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 14:54:06.522782 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 14:54:06.523647 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 14:54:06.523647 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 14:54:06.523647 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 14:54:06.523647 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 14:54:06.523647 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:54:06.523647 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:54:06.523647 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:54:06.523647 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:54:06.523647 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:54:06.523647 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:54:06.523647 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:54:06.523647 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 26 14:54:06.523647 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:54:06.523647 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 26 14:54:06.523647 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:54:06.523647 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:54:06.524396 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:54:06.524396 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 26 14:54:06.524396 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:54:06.524396 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 26 14:54:06.524396 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 14:54:06.524396 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 14:54:06.524396 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 14:54:06.524396 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 14:54:06.524396 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 14:54:06.524396 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 14:54:06.524396 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 14:54:06.524396 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 26 14:54:06.524396 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 37/76] 10.222.0.46 () {56 vars in 1114 bytes} [Thu May 26 14:54:06 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:54:06.525008 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c63ce9-34c6-49f4-a151-f7725fb70217 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:06.532302 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8523dd89-bc67-4844-9eee-c47bdc303eb4 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:06.575979 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aee732b-2ee8-4617-9c85-017951cca53e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:06.576912 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aee732b-2ee8-4617-9c85-017951cca53e None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:06.577196 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aee732b-2ee8-4617-9c85-017951cca53e None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:06.577535 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aee732b-2ee8-4617-9c85-017951cca53e None admin] PATH_INFO: `/v3/users/70e4c37945d34ae18f6ac64fb237b230` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:06.595661 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aee732b-2ee8-4617-9c85-017951cca53e None admin] RBAC: Authorizing `identity:get_user(user_id=70e4c37945d34ae18f6ac64fb237b230)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:54:06.597145 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aee732b-2ee8-4617-9c85-017951cca53e None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:54:06.598907 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 40/77] 10.222.0.46 () {56 vars in 1177 bytes} [Thu May 26 14:54:06 2022] GET /identity/v3/users/70e4c37945d34ae18f6ac64fb237b230 => generated 275 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:54:06.600343 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bdfe01-c5e9-4341-a1c9-ea3831619bee None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:06.600545 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bdfe01-c5e9-4341-a1c9-ea3831619bee None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:06.600848 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bdfe01-c5e9-4341-a1c9-ea3831619bee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:06.623435 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-65bdfe01-c5e9-4341-a1c9-ea3831619bee None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:54:06.623435 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-65bdfe01-c5e9-4341-a1c9-ea3831619bee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:54:06.624154 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:06.653412 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c63ce9-34c6-49f4-a151-f7725fb70217 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:06.675752 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:06.694615 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 41/78] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 14:54:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 14:54:06.695504 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bdfe01-c5e9-4341-a1c9-ea3831619bee None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:06.697058 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d36eac-a824-4dd7-bebc-af57203b5577 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:06.697989 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d36eac-a824-4dd7-bebc-af57203b5577 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:06.698372 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d36eac-a824-4dd7-bebc-af57203b5577 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:06.699385 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d36eac-a824-4dd7-bebc-af57203b5577 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:06.798962 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d36eac-a824-4dd7-bebc-af57203b5577 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:54:06.800462 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d36eac-a824-4dd7-bebc-af57203b5577 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:54:06.816583 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bdfe01-c5e9-4341-a1c9-ea3831619bee None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:06.858800 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59aad93-c308-4c21-a417-71e7f4a42cb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:06.859704 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59aad93-c308-4c21-a417-71e7f4a42cb2 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:06.860033 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59aad93-c308-4c21-a417-71e7f4a42cb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:06.860425 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59aad93-c308-4c21-a417-71e7f4a42cb2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:06.874972 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59aad93-c308-4c21-a417-71e7f4a42cb2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 14:54:06.884350 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-f59aad93-c308-4c21-a417-71e7f4a42cb2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 26 14:54:06.884350 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:54:06.884350 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:54:06.884350 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:54:06.884350 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:54:06.884350 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:54:06.884350 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:54:06.884350 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:54:06.884350 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:54:06.884350 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:54:06.884350 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:54:06.884350 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:54:06.884350 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 14:54:06.884350 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 14:54:06.884350 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 14:54:06.884350 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 14:54:06.885496 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 14:54:06.885496 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 14:54:06.885496 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 14:54:06.885496 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 14:54:06.885496 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:54:06.885496 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:54:06.885496 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:54:06.885496 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:54:06.885496 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:54:06.885496 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:54:06.885496 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:54:06.885496 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 26 14:54:06.885496 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:54:06.885496 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 26 14:54:06.885496 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:54:06.885496 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:54:06.886921 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:54:06.886921 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 26 14:54:06.886921 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:54:06.886921 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 26 14:54:06.886921 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 14:54:06.886921 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 14:54:06.886921 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 14:54:06.886921 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 14:54:06.886921 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 14:54:06.886921 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 14:54:06.886921 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 26 14:54:06.886921 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 26 14:54:06.886921 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 42/79] 10.222.0.46 () {56 vars in 1099 bytes} [Thu May 26 14:54:06 2022] GET /identity/v3/roles/member => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:54:06.886921 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59aad93-c308-4c21-a417-71e7f4a42cb2 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:06.910087 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:06.939588 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 38/80] 10.222.0.46 () {58 vars in 1282 bytes} [Thu May 26 14:54:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 14:54:06.939772 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:06.940442 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d36eac-a824-4dd7-bebc-af57203b5577 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:07.009760 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59aad93-c308-4c21-a417-71e7f4a42cb2 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:07.044004 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d36eac-a824-4dd7-bebc-af57203b5577 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:07.052586 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bf9052-1223-496d-9369-7cbb7a1f7c28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:07.053369 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bf9052-1223-496d-9369-7cbb7a1f7c28 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:07.053790 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bf9052-1223-496d-9369-7cbb7a1f7c28 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:07.054161 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bf9052-1223-496d-9369-7cbb7a1f7c28 None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:07.055105 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20bf9052-1223-496d-9369-7cbb7a1f7c28 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:54:07.055486 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20bf9052-1223-496d-9369-7cbb7a1f7c28 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:54:07.057190 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20bf9052-1223-496d-9369-7cbb7a1f7c28 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:54:07.068897 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 43/81] 10.222.0.46 () {56 vars in 1111 bytes} [Thu May 26 14:54:06 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:54:07.069896 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bf9052-1223-496d-9369-7cbb7a1f7c28 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:07.086371 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a023149-6d4e-4ceb-88ea-d9e362d5844f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:07.087035 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a023149-6d4e-4ceb-88ea-d9e362d5844f None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:07.087321 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a023149-6d4e-4ceb-88ea-d9e362d5844f None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:07.087606 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a023149-6d4e-4ceb-88ea-d9e362d5844f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:07.132124 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:07.160429 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a023149-6d4e-4ceb-88ea-d9e362d5844f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:54:07.162227 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a023149-6d4e-4ceb-88ea-d9e362d5844f None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:54:07.189783 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bf9052-1223-496d-9369-7cbb7a1f7c28 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:07.197174 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:07.236899 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f758d965-6ad7-40b7-8a0c-1c027e51544e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:07.237885 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f758d965-6ad7-40b7-8a0c-1c027e51544e None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:07.238246 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f758d965-6ad7-40b7-8a0c-1c027e51544e None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:07.238609 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f758d965-6ad7-40b7-8a0c-1c027e51544e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:07.240008 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f758d965-6ad7-40b7-8a0c-1c027e51544e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:54:07.241902 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f758d965-6ad7-40b7-8a0c-1c027e51544e None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:54:07.243996 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 44/82] 10.222.0.46 () {56 vars in 1108 bytes} [Thu May 26 14:54:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:54:07.245260 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f758d965-6ad7-40b7-8a0c-1c027e51544e None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:07.293681 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 39/83] 10.222.0.46 () {58 vars in 1282 bytes} [Thu May 26 14:54:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 14:54:07.295879 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a023149-6d4e-4ceb-88ea-d9e362d5844f None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:07.373360 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:07.378103 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f758d965-6ad7-40b7-8a0c-1c027e51544e None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:07.402061 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a023149-6d4e-4ceb-88ea-d9e362d5844f None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:07.420393 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200d846e-81a0-4243-9a96-8697c3985fb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:07.421316 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200d846e-81a0-4243-9a96-8697c3985fb8 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:07.421819 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200d846e-81a0-4243-9a96-8697c3985fb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:07.421932 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200d846e-81a0-4243-9a96-8697c3985fb8 None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:07.422755 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200d846e-81a0-4243-9a96-8697c3985fb8 None admin] RBAC: Adding query filter params (name=member) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:54:07.423155 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200d846e-81a0-4243-9a96-8697c3985fb8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:54:07.424391 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200d846e-81a0-4243-9a96-8697c3985fb8 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:54:07.433532 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 45/84] 10.222.0.46 () {56 vars in 1101 bytes} [Thu May 26 14:54:07 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:54:07.435078 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200d846e-81a0-4243-9a96-8697c3985fb8 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:07.445383 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5071fbf-54e8-44fb-ab7e-85384b234a8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:07.446416 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5071fbf-54e8-44fb-ab7e-85384b234a8f None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:07.446748 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5071fbf-54e8-44fb-ab7e-85384b234a8f None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:07.447078 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5071fbf-54e8-44fb-ab7e-85384b234a8f None admin] PATH_INFO: `/v3/projects/e474420e5c124a3f86383fce1ef4c4fb` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:07.448755 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5071fbf-54e8-44fb-ab7e-85384b234a8f None admin] RBAC: Authorizing `identity:get_project(project_id=e474420e5c124a3f86383fce1ef4c4fb)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:54:07.450468 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5071fbf-54e8-44fb-ab7e-85384b234a8f None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:54:07.452443 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 40/85] 10.222.0.46 () {56 vars in 1186 bytes} [Thu May 26 14:54:07 2022] GET /identity/v3/projects/e474420e5c124a3f86383fce1ef4c4fb => generated 298 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:54:07.453313 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5071fbf-54e8-44fb-ab7e-85384b234a8f None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:07.477108 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:07.553143 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200d846e-81a0-4243-9a96-8697c3985fb8 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:07.580568 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5071fbf-54e8-44fb-ab7e-85384b234a8f None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:07.594355 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9678de3d-c5c9-40bd-932a-b764e1d0bf76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:07.595529 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9678de3d-c5c9-40bd-932a-b764e1d0bf76 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:07.595870 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9678de3d-c5c9-40bd-932a-b764e1d0bf76 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:07.596240 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9678de3d-c5c9-40bd-932a-b764e1d0bf76 None admin] PATH_INFO: `/v3/users/70e4c37945d34ae18f6ac64fb237b230` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:07.597935 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9678de3d-c5c9-40bd-932a-b764e1d0bf76 None admin] RBAC: Authorizing `identity:get_user(user_id=70e4c37945d34ae18f6ac64fb237b230)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:54:07.600162 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9678de3d-c5c9-40bd-932a-b764e1d0bf76 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:54:07.607973 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 46/86] 10.222.0.46 () {56 vars in 1177 bytes} [Thu May 26 14:54:07 2022] GET /identity/v3/users/70e4c37945d34ae18f6ac64fb237b230 => generated 275 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:54:07.607973 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9678de3d-c5c9-40bd-932a-b764e1d0bf76 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:07.624778 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0ef621-d25e-4d48-9b60-b826b9a70c78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:07.626196 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0ef621-d25e-4d48-9b60-b826b9a70c78 None admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:07.626487 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0ef621-d25e-4d48-9b60-b826b9a70c78 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:07.626771 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0ef621-d25e-4d48-9b60-b826b9a70c78 None admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:07.627496 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0ef621-d25e-4d48-9b60-b826b9a70c78 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:54:07.628994 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0ef621-d25e-4d48-9b60-b826b9a70c78 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:54:07.644234 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:07.650793 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 41/87] 10.222.0.46 () {60 vars in 1132 bytes} [Thu May 26 14:54:07 2022] POST /identity/v3/users => generated 289 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 14:54:07.665655 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0ef621-d25e-4d48-9b60-b826b9a70c78 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:07.746101 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9678de3d-c5c9-40bd-932a-b764e1d0bf76 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:07.750401 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:07.801217 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56f07bb-69c5-4217-8818-e163648eb25e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:07.802195 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56f07bb-69c5-4217-8818-e163648eb25e None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:07.802656 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56f07bb-69c5-4217-8818-e163648eb25e None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:07.802869 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56f07bb-69c5-4217-8818-e163648eb25e None admin] PATH_INFO: `/v3/projects/e474420e5c124a3f86383fce1ef4c4fb` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:07.804168 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56f07bb-69c5-4217-8818-e163648eb25e None admin] RBAC: Authorizing `identity:get_project(project_id=e474420e5c124a3f86383fce1ef4c4fb)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:54:07.805798 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56f07bb-69c5-4217-8818-e163648eb25e None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:54:07.814253 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 47/88] 10.222.0.46 () {56 vars in 1186 bytes} [Thu May 26 14:54:07 2022] GET /identity/v3/projects/e474420e5c124a3f86383fce1ef4c4fb => generated 298 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:54:07.814253 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56f07bb-69c5-4217-8818-e163648eb25e None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:07.826724 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0ef621-d25e-4d48-9b60-b826b9a70c78 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:07.880168 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143c4ca1-5812-466b-a165-bb5c676169b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:07.880931 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143c4ca1-5812-466b-a165-bb5c676169b7 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:07.881214 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143c4ca1-5812-466b-a165-bb5c676169b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:07.881512 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143c4ca1-5812-466b-a165-bb5c676169b7 None admin] PATH_INFO: `/v3/users/70e4c37945d34ae18f6ac64fb237b230` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:07.883010 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143c4ca1-5812-466b-a165-bb5c676169b7 None admin] RBAC: Authorizing `identity:get_user(user_id=70e4c37945d34ae18f6ac64fb237b230)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:54:07.884257 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143c4ca1-5812-466b-a165-bb5c676169b7 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:54:07.891302 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 42/89] 10.222.0.46 () {56 vars in 1177 bytes} [Thu May 26 14:54:07 2022] GET /identity/v3/users/70e4c37945d34ae18f6ac64fb237b230 => generated 275 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:54:07.891302 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143c4ca1-5812-466b-a165-bb5c676169b7 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:07.971565 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56f07bb-69c5-4217-8818-e163648eb25e None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:07.974516 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:08.027653 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143c4ca1-5812-466b-a165-bb5c676169b7 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:08.041613 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0c1e71-2abf-4c6d-9d66-da953cbf9b84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:08.042615 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0c1e71-2abf-4c6d-9d66-da953cbf9b84 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:08.043370 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0c1e71-2abf-4c6d-9d66-da953cbf9b84 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:08.044579 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0c1e71-2abf-4c6d-9d66-da953cbf9b84 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:08.045962 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0c1e71-2abf-4c6d-9d66-da953cbf9b84 None admin] RBAC: Adding query filter params (role.id=a2e551773cbd4df084020676865b8222, scope.project.id=e474420e5c124a3f86383fce1ef4c4fb, user.id=dede0ef3af5b453796bce68732b5fd29) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:54:08.046306 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0c1e71-2abf-4c6d-9d66-da953cbf9b84 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:54:08.048442 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0c1e71-2abf-4c6d-9d66-da953cbf9b84 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:54:08.061188 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 48/90] 10.222.0.46 () {56 vars in 1374 bytes} [Thu May 26 14:54:07 2022] GET /identity/v3/role_assignments?user.id=dede0ef3af5b453796bce68732b5fd29&scope.project.id=e474420e5c124a3f86383fce1ef4c4fb&role.id=a2e551773cbd4df084020676865b8222 => generated 259 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:54:08.062484 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0c1e71-2abf-4c6d-9d66-da953cbf9b84 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:08.107918 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:08.111927 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0947411e-baca-46de-ba59-104e1da803da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:08.112736 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0947411e-baca-46de-ba59-104e1da803da None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:08.113035 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0947411e-baca-46de-ba59-104e1da803da None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:08.113481 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0947411e-baca-46de-ba59-104e1da803da None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:08.230292 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0c1e71-2abf-4c6d-9d66-da953cbf9b84 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:08.250985 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0947411e-baca-46de-ba59-104e1da803da None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:54:08.252550 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0947411e-baca-46de-ba59-104e1da803da None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:54:08.312854 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8357232-fd9b-46d2-9b7e-532fd2098d99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:08.315815 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8357232-fd9b-46d2-9b7e-532fd2098d99 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:08.316138 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8357232-fd9b-46d2-9b7e-532fd2098d99 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:08.316425 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8357232-fd9b-46d2-9b7e-532fd2098d99 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:08.317307 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8357232-fd9b-46d2-9b7e-532fd2098d99 None admin] RBAC: Adding query filter params (role.id=d4088d1845a24fbd9bd8975ce0b1d3d0, scope.project.id=e474420e5c124a3f86383fce1ef4c4fb, user.id=70e4c37945d34ae18f6ac64fb237b230) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:54:08.317658 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8357232-fd9b-46d2-9b7e-532fd2098d99 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:54:08.319202 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8357232-fd9b-46d2-9b7e-532fd2098d99 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:54:08.335161 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 49/91] 10.222.0.46 () {56 vars in 1374 bytes} [Thu May 26 14:54:08 2022] GET /identity/v3/role_assignments?user.id=70e4c37945d34ae18f6ac64fb237b230&scope.project.id=e474420e5c124a3f86383fce1ef4c4fb&role.id=d4088d1845a24fbd9bd8975ce0b1d3d0 => generated 259 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:54:08.335820 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8357232-fd9b-46d2-9b7e-532fd2098d99 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:08.395012 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:08.468914 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:08.472991 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8357232-fd9b-46d2-9b7e-532fd2098d99 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:08.490814 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 43/92] 10.222.0.46 () {58 vars in 1282 bytes} [Thu May 26 14:54:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 604 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 14:54:08.498729 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0947411e-baca-46de-ba59-104e1da803da None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:08.516506 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a8a94a-414a-4914-8cfc-dce6a3c27cec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:08.517186 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a8a94a-414a-4914-8cfc-dce6a3c27cec None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:08.518951 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a8a94a-414a-4914-8cfc-dce6a3c27cec None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:08.519275 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a8a94a-414a-4914-8cfc-dce6a3c27cec None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:08.679668 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a8a94a-414a-4914-8cfc-dce6a3c27cec None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:54:08.681202 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a8a94a-414a-4914-8cfc-dce6a3c27cec None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:54:08.717355 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0947411e-baca-46de-ba59-104e1da803da None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:08.796092 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc36dd4-d54c-412c-9f59-20e5299a5fbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:08.796979 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc36dd4-d54c-412c-9f59-20e5299a5fbb None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:08.797264 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc36dd4-d54c-412c-9f59-20e5299a5fbb None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:08.797573 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc36dd4-d54c-412c-9f59-20e5299a5fbb None admin] PATH_INFO: `/v3/projects/e474420e5c124a3f86383fce1ef4c4fb` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:08.802425 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc36dd4-d54c-412c-9f59-20e5299a5fbb None admin] RBAC: Authorizing `identity:get_project(project_id=e474420e5c124a3f86383fce1ef4c4fb)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:54:08.803934 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc36dd4-d54c-412c-9f59-20e5299a5fbb None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:54:08.811205 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 44/93] 10.222.0.46 () {56 vars in 1186 bytes} [Thu May 26 14:54:08 2022] GET /identity/v3/projects/e474420e5c124a3f86383fce1ef4c4fb => generated 298 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 14:54:08.817306 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc36dd4-d54c-412c-9f59-20e5299a5fbb None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:08.883964 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:08.893887 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:08.894771 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 50/94] 10.222.0.46 () {58 vars in 1282 bytes} [Thu May 26 14:54:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 559 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 14:54:08.895990 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a8a94a-414a-4914-8cfc-dce6a3c27cec None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:09.017210 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc36dd4-d54c-412c-9f59-20e5299a5fbb None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:09.078921 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c728a171-55e5-45f0-95ac-9f8793ca79c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:09.080126 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c728a171-55e5-45f0-95ac-9f8793ca79c9 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:09.080709 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c728a171-55e5-45f0-95ac-9f8793ca79c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:09.081178 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c728a171-55e5-45f0-95ac-9f8793ca79c9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:09.082250 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c728a171-55e5-45f0-95ac-9f8793ca79c9 None admin] RBAC: Adding query filter params (role.id=160115b2143c4b389d39cd93f43e1d01, scope.project.id=e474420e5c124a3f86383fce1ef4c4fb, user.id=70e4c37945d34ae18f6ac64fb237b230) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:54:09.082773 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c728a171-55e5-45f0-95ac-9f8793ca79c9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:54:09.084635 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c728a171-55e5-45f0-95ac-9f8793ca79c9 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:54:09.098429 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 45/95] 10.222.0.46 () {56 vars in 1374 bytes} [Thu May 26 14:54:08 2022] GET /identity/v3/role_assignments?user.id=70e4c37945d34ae18f6ac64fb237b230&scope.project.id=e474420e5c124a3f86383fce1ef4c4fb&role.id=160115b2143c4b389d39cd93f43e1d01 => generated 259 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 14:54:09.152800 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a8a94a-414a-4914-8cfc-dce6a3c27cec None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:09.258511 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-735b71be-fd47-45f4-ae25-61c122c565de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:09.259586 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735b71be-fd47-45f4-ae25-61c122c565de None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:09.259934 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735b71be-fd47-45f4-ae25-61c122c565de None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:09.260272 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-735b71be-fd47-45f4-ae25-61c122c565de None admin] PATH_INFO: `/v3/projects/1c95d1c3fe20488caf82b4b1125bfbc1` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:09.261901 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735b71be-fd47-45f4-ae25-61c122c565de None admin] RBAC: Authorizing `identity:get_project(project_id=1c95d1c3fe20488caf82b4b1125bfbc1)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:54:09.263640 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-735b71be-fd47-45f4-ae25-61c122c565de None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:54:09.265670 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 51/96] 10.222.0.46 () {56 vars in 1186 bytes} [Thu May 26 14:54:08 2022] GET /identity/v3/projects/1c95d1c3fe20488caf82b4b1125bfbc1 => generated 312 bytes in 370 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:54:09.271814 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c728a171-55e5-45f0-95ac-9f8793ca79c9 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:09.429921 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:09.435765 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c728a171-55e5-45f0-95ac-9f8793ca79c9 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:09.494242 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134c9587-d769-4da9-b0a5-7ce375181c7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:09.495327 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134c9587-d769-4da9-b0a5-7ce375181c7c None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:09.495829 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134c9587-d769-4da9-b0a5-7ce375181c7c None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:09.496324 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134c9587-d769-4da9-b0a5-7ce375181c7c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:09.497434 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134c9587-d769-4da9-b0a5-7ce375181c7c None admin] RBAC: Adding query filter params (role.id=d4088d1845a24fbd9bd8975ce0b1d3d0, scope.project.id=1c95d1c3fe20488caf82b4b1125bfbc1, user.id=70e4c37945d34ae18f6ac64fb237b230) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:54:09.497986 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134c9587-d769-4da9-b0a5-7ce375181c7c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:54:09.499685 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134c9587-d769-4da9-b0a5-7ce375181c7c None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:54:09.507820 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 46/97] 10.222.0.46 () {56 vars in 1374 bytes} [Thu May 26 14:54:09 2022] GET /identity/v3/role_assignments?user.id=70e4c37945d34ae18f6ac64fb237b230&scope.project.id=1c95d1c3fe20488caf82b4b1125bfbc1&role.id=d4088d1845a24fbd9bd8975ce0b1d3d0 => generated 259 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 14:54:11.134545 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee05879-c0da-458b-ab18-76c04f4cf4e9 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:11.135667 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee05879-c0da-458b-ab18-76c04f4cf4e9 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:11.136169 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee05879-c0da-458b-ab18-76c04f4cf4e9 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:11.138951 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 52/98] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 14:54:11 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 14:54:11.145942 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cbdb69-d997-4677-b923-de60bda09264 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:11.146879 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cbdb69-d997-4677-b923-de60bda09264 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:11.147401 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cbdb69-d997-4677-b923-de60bda09264 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:11.166201 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-31cbdb69-d997-4677-b923-de60bda09264 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:54:11.167797 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-31cbdb69-d997-4677-b923-de60bda09264 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:54:11.244131 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 47/99] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 14:54:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 14:54:11.274297 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee05879-c0da-458b-ab18-76c04f4cf4e9 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:11.370970 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:11.424050 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee05879-c0da-458b-ab18-76c04f4cf4e9 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:11.475332 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae802922-4184-4f94-8679-81f3e0af5938 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:11.476222 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae802922-4184-4f94-8679-81f3e0af5938 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:11.476590 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae802922-4184-4f94-8679-81f3e0af5938 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:11.476893 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae802922-4184-4f94-8679-81f3e0af5938 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:11.488654 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae802922-4184-4f94-8679-81f3e0af5938 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 14:54:11.497621 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-ae802922-4184-4f94-8679-81f3e0af5938 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 26 14:54:11.497621 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:54:11.497621 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:54:11.497621 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:54:11.497621 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:54:11.497621 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:54:11.497621 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:54:11.497621 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:54:11.497621 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:54:11.497621 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:54:11.497621 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:54:11.497621 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:54:11.497621 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 14:54:11.497621 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 14:54:11.497621 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 14:54:11.497621 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 14:54:11.498528 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 14:54:11.498528 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 14:54:11.498528 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 14:54:11.498528 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 14:54:11.498528 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:54:11.498528 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:54:11.498528 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:54:11.498528 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:54:11.498528 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:54:11.498528 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:54:11.498528 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:54:11.498528 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 26 14:54:11.498528 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:54:11.498528 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 26 14:54:11.498528 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:54:11.498528 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:54:11.499345 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:54:11.499345 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 26 14:54:11.499345 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:54:11.499345 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 26 14:54:11.499345 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 14:54:11.499345 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 14:54:11.499345 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 14:54:11.499345 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 14:54:11.499345 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 14:54:11.499345 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 14:54:11.499345 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 26 14:54:11.499345 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 26 14:54:11.499345 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 53/100] 10.222.0.46 () {56 vars in 1099 bytes} [Thu May 26 14:54:11 2022] GET /identity/v3/roles/reader => generated 84 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:54:11.504859 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cbdb69-d997-4677-b923-de60bda09264 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:11.608461 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756d7e2a-5794-48bd-b880-863245060596 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:11.608461 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756d7e2a-5794-48bd-b880-863245060596 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:11.608461 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756d7e2a-5794-48bd-b880-863245060596 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:11.609126 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 54/101] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 14:54:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 14:54:11.624954 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70f0a70-8417-470b-86e6-d8b46ff9a129 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:11.624954 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70f0a70-8417-470b-86e6-d8b46ff9a129 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:11.624954 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70f0a70-8417-470b-86e6-d8b46ff9a129 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:11.667778 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-d70f0a70-8417-470b-86e6-d8b46ff9a129 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:54:11.669080 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-d70f0a70-8417-470b-86e6-d8b46ff9a129 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:54:11.681955 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:11.695110 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cbdb69-d997-4677-b923-de60bda09264 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:11.789417 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c0a645-8524-43f8-a64a-304992d2ade7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:11.790179 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c0a645-8524-43f8-a64a-304992d2ade7 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:11.790513 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c0a645-8524-43f8-a64a-304992d2ade7 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:11.790852 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c0a645-8524-43f8-a64a-304992d2ade7 None admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:11.791646 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c0a645-8524-43f8-a64a-304992d2ade7 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:54:11.792011 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c0a645-8524-43f8-a64a-304992d2ade7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:54:11.793797 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c0a645-8524-43f8-a64a-304992d2ade7 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:54:11.804165 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 55/102] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 14:54:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 14:54:11.808909 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 48/103] 10.222.0.46 () {56 vars in 1101 bytes} [Thu May 26 14:54:11 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:54:11.815537 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70f0a70-8417-470b-86e6-d8b46ff9a129 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:11.846977 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c0a645-8524-43f8-a64a-304992d2ade7 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:11.982407 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:12.048621 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c0a645-8524-43f8-a64a-304992d2ade7 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:12.067168 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70f0a70-8417-470b-86e6-d8b46ff9a129 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:12.097203 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7099d078-23b4-4ba4-96ed-5089871f6849 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:12.098148 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7099d078-23b4-4ba4-96ed-5089871f6849 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:12.098435 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7099d078-23b4-4ba4-96ed-5089871f6849 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:12.098728 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7099d078-23b4-4ba4-96ed-5089871f6849 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:12.100290 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7099d078-23b4-4ba4-96ed-5089871f6849 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:54:12.102004 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7099d078-23b4-4ba4-96ed-5089871f6849 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:54:12.103963 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 49/104] 10.222.0.46 () {56 vars in 1108 bytes} [Thu May 26 14:54:11 2022] GET /identity/v3/domains/default => generated 199 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 14:54:12.105078 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f746224-2266-4115-8f0b-c099a649cbb6 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:12.105748 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f746224-2266-4115-8f0b-c099a649cbb6 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:12.106201 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f746224-2266-4115-8f0b-c099a649cbb6 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:12.108423 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 50/105] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 14:54:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 14:54:12.110115 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92b5ed7-24a0-485f-b097-cbcc02c94762 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:12.110590 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92b5ed7-24a0-485f-b097-cbcc02c94762 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:12.111062 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92b5ed7-24a0-485f-b097-cbcc02c94762 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:12.113619 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 51/106] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 14:54:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 14:54:12.115253 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53c8c8f-8789-4c07-8005-09da3fc29311 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:12.115621 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53c8c8f-8789-4c07-8005-09da3fc29311 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:12.116035 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53c8c8f-8789-4c07-8005-09da3fc29311 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:12.125060 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de3893c-2cdc-4068-b6e1-3f149decb558 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:12.126345 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de3893c-2cdc-4068-b6e1-3f149decb558 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:12.126851 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de3893c-2cdc-4068-b6e1-3f149decb558 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:12.127360 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de3893c-2cdc-4068-b6e1-3f149decb558 None admin] PATH_INFO: `/v3/users/990e3996e18d4a1aa532413cc2dbc3a1` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:12.139004 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-f53c8c8f-8789-4c07-8005-09da3fc29311 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:54:12.140261 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-f53c8c8f-8789-4c07-8005-09da3fc29311 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:54:12.143861 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de3893c-2cdc-4068-b6e1-3f149decb558 None admin] RBAC: Authorizing `identity:get_user(user_id=990e3996e18d4a1aa532413cc2dbc3a1)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:54:12.146295 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de3893c-2cdc-4068-b6e1-3f149decb558 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:54:12.148283 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 56/107] 10.222.0.46 () {56 vars in 1177 bytes} [Thu May 26 14:54:11 2022] GET /identity/v3/users/990e3996e18d4a1aa532413cc2dbc3a1 => generated 289 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:54:12.149336 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de3893c-2cdc-4068-b6e1-3f149decb558 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:12.165031 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:12.240374 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 52/108] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 14:54:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 14:54:12.243671 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d8e7e4-b230-4fc7-ad53-8b8675efee08 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:12.244434 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d8e7e4-b230-4fc7-ad53-8b8675efee08 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:12.245052 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d8e7e4-b230-4fc7-ad53-8b8675efee08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:12.271603 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-c7d8e7e4-b230-4fc7-ad53-8b8675efee08 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:54:12.273307 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-c7d8e7e4-b230-4fc7-ad53-8b8675efee08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:54:12.298177 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de3893c-2cdc-4068-b6e1-3f149decb558 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:12.319980 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:12.344008 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6de5be-2eea-42b2-ada7-125011c7fa58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:12.344769 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6de5be-2eea-42b2-ada7-125011c7fa58 None admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:12.345110 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6de5be-2eea-42b2-ada7-125011c7fa58 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:12.345395 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6de5be-2eea-42b2-ada7-125011c7fa58 None admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:12.346405 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6de5be-2eea-42b2-ada7-125011c7fa58 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:54:12.348024 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6de5be-2eea-42b2-ada7-125011c7fa58 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:54:12.364941 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 53/109] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 14:54:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 14:54:12.375283 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d8e7e4-b230-4fc7-ad53-8b8675efee08 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:12.385691 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 57/110] 10.222.0.46 () {60 vars in 1141 bytes} [Thu May 26 14:54:12 2022] POST /identity/v3/projects => generated 302 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 14:54:12.390333 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6de5be-2eea-42b2-ada7-125011c7fa58 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:12.550660 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6de5be-2eea-42b2-ada7-125011c7fa58 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:12.595905 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:12.615034 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9336a56-c28e-4e0a-9a50-50ba4205fbad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:12.617138 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9336a56-c28e-4e0a-9a50-50ba4205fbad None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:12.617751 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9336a56-c28e-4e0a-9a50-50ba4205fbad None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:12.618263 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9336a56-c28e-4e0a-9a50-50ba4205fbad None admin] PATH_INFO: `/v3/roles/member` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:12.627265 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d8e7e4-b230-4fc7-ad53-8b8675efee08 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:12.646592 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9336a56-c28e-4e0a-9a50-50ba4205fbad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 14:54:12.660707 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-f9336a56-c28e-4e0a-9a50-50ba4205fbad None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 26 14:54:12.660707 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:54:12.660707 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:54:12.660707 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:54:12.660707 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:54:12.660707 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:54:12.660707 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:54:12.660707 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:54:12.660707 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:54:12.660707 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:54:12.660707 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:54:12.660707 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:54:12.660707 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 14:54:12.660707 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 14:54:12.660707 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 14:54:12.660707 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 14:54:12.662154 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 14:54:12.662154 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 14:54:12.662154 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 14:54:12.662154 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 14:54:12.662154 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:54:12.662154 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:54:12.662154 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:54:12.662154 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:54:12.662154 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:54:12.662154 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:54:12.662154 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:54:12.662154 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 26 14:54:12.662154 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:54:12.662154 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 26 14:54:12.662154 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:54:12.662154 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:54:12.662957 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:54:12.662957 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 26 14:54:12.662957 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:54:12.662957 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 26 14:54:12.662957 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 14:54:12.662957 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 14:54:12.662957 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 14:54:12.662957 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 14:54:12.662957 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 14:54:12.662957 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 14:54:12.662957 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 26 14:54:12.662957 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 26 14:54:12.663745 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 58/111] 10.222.0.46 () {56 vars in 1099 bytes} [Thu May 26 14:54:12 2022] GET /identity/v3/roles/member => generated 84 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:54:12.665038 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9336a56-c28e-4e0a-9a50-50ba4205fbad None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:12.688999 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaef574b-6693-479f-b531-52b0a8897adb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:12.689790 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaef574b-6693-479f-b531-52b0a8897adb None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:12.690080 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaef574b-6693-479f-b531-52b0a8897adb None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:12.690368 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaef574b-6693-479f-b531-52b0a8897adb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:12.718098 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:12.794606 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaef574b-6693-479f-b531-52b0a8897adb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:54:12.796937 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaef574b-6693-479f-b531-52b0a8897adb None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:54:12.832341 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9336a56-c28e-4e0a-9a50-50ba4205fbad None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:12.874939 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55f8084-edc3-4754-b807-7595e4ff09ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:12.875764 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55f8084-edc3-4754-b807-7595e4ff09ed None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:12.876054 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55f8084-edc3-4754-b807-7595e4ff09ed None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:12.876338 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55f8084-edc3-4754-b807-7595e4ff09ed None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:12.889155 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55f8084-edc3-4754-b807-7595e4ff09ed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 14:54:12.897773 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-f55f8084-edc3-4754-b807-7595e4ff09ed None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 14:54:12.897773 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:54:12.897773 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:54:12.897773 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:54:12.897773 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:54:12.897773 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:54:12.897773 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:54:12.897773 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:54:12.897773 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:54:12.897773 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:54:12.897773 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:54:12.897773 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:54:12.897773 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 14:54:12.897773 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 14:54:12.897773 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 14:54:12.897773 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 14:54:12.899954 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 14:54:12.899954 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 14:54:12.899954 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 14:54:12.899954 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 14:54:12.899954 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:54:12.899954 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:54:12.899954 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:54:12.899954 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:54:12.899954 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:54:12.899954 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:54:12.899954 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:54:12.899954 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 26 14:54:12.899954 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:54:12.899954 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 26 14:54:12.899954 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:54:12.899954 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:54:12.901880 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:54:12.901880 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 26 14:54:12.901880 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:54:12.901880 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 26 14:54:12.901880 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 14:54:12.901880 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 14:54:12.901880 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 14:54:12.901880 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 14:54:12.901880 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 14:54:12.901880 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 14:54:12.901880 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 14:54:12.901880 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 26 14:54:12.901880 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 59/112] 10.222.0.46 () {56 vars in 1096 bytes} [Thu May 26 14:54:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:54:12.909757 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6802d666-a991-466c-8a07-57424e76559b None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:12.909757 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6802d666-a991-466c-8a07-57424e76559b None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:12.909757 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6802d666-a991-466c-8a07-57424e76559b None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:12.909757 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 60/113] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 14:54:12 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 14:54:12.909757 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6802d666-a991-466c-8a07-57424e76559b None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:12.939972 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:12.968852 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 54/114] 10.222.0.46 () {58 vars in 1282 bytes} [Thu May 26 14:54:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 603 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 14:54:12.970383 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaef574b-6693-479f-b531-52b0a8897adb None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:13.030798 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:13.053125 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6802d666-a991-466c-8a07-57424e76559b None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:13.108105 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9edebf-b9d2-4da6-a823-4939117b190d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:13.109134 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9edebf-b9d2-4da6-a823-4939117b190d None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:13.109626 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9edebf-b9d2-4da6-a823-4939117b190d None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:13.110065 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9edebf-b9d2-4da6-a823-4939117b190d None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:13.111269 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9edebf-b9d2-4da6-a823-4939117b190d None admin] RBAC: Adding query filter params (name=member) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:54:13.112230 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9edebf-b9d2-4da6-a823-4939117b190d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:54:13.114408 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d9edebf-b9d2-4da6-a823-4939117b190d None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:54:13.125239 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaef574b-6693-479f-b531-52b0a8897adb None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:13.128005 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 61/115] 10.222.0.46 () {56 vars in 1101 bytes} [Thu May 26 14:54:12 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:54:13.130837 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e3c1f4-7e30-410c-9f4a-9d094cfe1d96 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:13.131303 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e3c1f4-7e30-410c-9f4a-9d094cfe1d96 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:13.131751 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e3c1f4-7e30-410c-9f4a-9d094cfe1d96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:13.156023 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-17e3c1f4-7e30-410c-9f4a-9d094cfe1d96 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:54:13.157944 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-17e3c1f4-7e30-410c-9f4a-9d094cfe1d96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:54:13.175097 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70d1828-4896-45b1-94ac-162d7e1498df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:13.175692 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70d1828-4896-45b1-94ac-162d7e1498df None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:13.175944 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70d1828-4896-45b1-94ac-162d7e1498df None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:13.176225 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70d1828-4896-45b1-94ac-162d7e1498df None admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:13.176994 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70d1828-4896-45b1-94ac-162d7e1498df None admin] RBAC: Adding query filter params (name=admin) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:54:13.177308 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70d1828-4896-45b1-94ac-162d7e1498df None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:54:13.179006 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70d1828-4896-45b1-94ac-162d7e1498df None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:54:13.194192 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 55/116] 10.222.0.46 () {56 vars in 1099 bytes} [Thu May 26 14:54:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:54:13.195325 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70d1828-4896-45b1-94ac-162d7e1498df None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:13.255557 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 62/117] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 14:54:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 14:54:13.256777 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e3c1f4-7e30-410c-9f4a-9d094cfe1d96 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:13.267399 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:13.341838 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70d1828-4896-45b1-94ac-162d7e1498df None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:13.368529 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:13.379510 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5d5ba2-64eb-46ca-89cb-69f88008abc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:13.381100 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5d5ba2-64eb-46ca-89cb-69f88008abc5 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:13.381396 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5d5ba2-64eb-46ca-89cb-69f88008abc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:13.381728 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5d5ba2-64eb-46ca-89cb-69f88008abc5 None admin] PATH_INFO: `/v3/projects/e474420e5c124a3f86383fce1ef4c4fb` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:13.383133 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5d5ba2-64eb-46ca-89cb-69f88008abc5 None admin] RBAC: Authorizing `identity:get_project(project_id=e474420e5c124a3f86383fce1ef4c4fb)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:54:13.384623 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5d5ba2-64eb-46ca-89cb-69f88008abc5 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:54:13.391513 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 56/118] 10.222.0.46 () {56 vars in 1186 bytes} [Thu May 26 14:54:13 2022] GET /identity/v3/projects/e474420e5c124a3f86383fce1ef4c4fb => generated 298 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:54:13.391513 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5d5ba2-64eb-46ca-89cb-69f88008abc5 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:13.401844 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e3c1f4-7e30-410c-9f4a-9d094cfe1d96 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:13.443031 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5da78f5-5da4-45ac-9e1a-0181fbab938d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:13.443925 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5da78f5-5da4-45ac-9e1a-0181fbab938d None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:13.444213 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5da78f5-5da4-45ac-9e1a-0181fbab938d None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:13.444495 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5da78f5-5da4-45ac-9e1a-0181fbab938d None admin] PATH_INFO: `/v3/users/70e4c37945d34ae18f6ac64fb237b230` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:13.445758 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5da78f5-5da4-45ac-9e1a-0181fbab938d None admin] RBAC: Authorizing `identity:get_user(user_id=70e4c37945d34ae18f6ac64fb237b230)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:54:13.447251 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5da78f5-5da4-45ac-9e1a-0181fbab938d None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:54:13.448927 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 63/119] 10.222.0.46 () {56 vars in 1177 bytes} [Thu May 26 14:54:13 2022] GET /identity/v3/users/70e4c37945d34ae18f6ac64fb237b230 => generated 275 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:54:13.450740 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88af3740-7dbe-4144-bb00-d8c684d5c1f9 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:13.451108 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88af3740-7dbe-4144-bb00-d8c684d5c1f9 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:13.451394 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88af3740-7dbe-4144-bb00-d8c684d5c1f9 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:13.456865 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 64/120] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 14:54:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 14:54:13.456865 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88af3740-7dbe-4144-bb00-d8c684d5c1f9 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:13.551572 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5d5ba2-64eb-46ca-89cb-69f88008abc5 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:13.566826 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:13.585821 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88af3740-7dbe-4144-bb00-d8c684d5c1f9 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:13.592863 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f9e948-c53d-4bb1-8446-c15a321d41e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:13.593818 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f9e948-c53d-4bb1-8446-c15a321d41e8 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:13.594121 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f9e948-c53d-4bb1-8446-c15a321d41e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:13.594412 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f9e948-c53d-4bb1-8446-c15a321d41e8 None admin] PATH_INFO: `/v3/users/dede0ef3af5b453796bce68732b5fd29` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:13.595643 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f9e948-c53d-4bb1-8446-c15a321d41e8 None admin] RBAC: Authorizing `identity:get_user(user_id=dede0ef3af5b453796bce68732b5fd29)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:54:13.597174 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f9e948-c53d-4bb1-8446-c15a321d41e8 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:54:13.605513 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 57/121] 10.222.0.46 () {56 vars in 1177 bytes} [Thu May 26 14:54:13 2022] GET /identity/v3/users/dede0ef3af5b453796bce68732b5fd29 => generated 247 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:54:13.605513 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f9e948-c53d-4bb1-8446-c15a321d41e8 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:13.634637 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4579ef0-d702-4de7-b0e1-6e4c7fb7c5c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:13.635775 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4579ef0-d702-4de7-b0e1-6e4c7fb7c5c8 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:13.636259 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4579ef0-d702-4de7-b0e1-6e4c7fb7c5c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:13.636733 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4579ef0-d702-4de7-b0e1-6e4c7fb7c5c8 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:13.649007 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4579ef0-d702-4de7-b0e1-6e4c7fb7c5c8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 14:54:13.658099 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-b4579ef0-d702-4de7-b0e1-6e4c7fb7c5c8 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 14:54:13.658099 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:54:13.658099 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:54:13.658099 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:54:13.658099 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:54:13.658099 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:54:13.658099 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:54:13.658099 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:54:13.658099 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:54:13.658099 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:54:13.658099 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:54:13.658099 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:54:13.658099 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 14:54:13.658099 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 14:54:13.658099 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 14:54:13.658099 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 14:54:13.658995 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 14:54:13.658995 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 14:54:13.658995 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 14:54:13.658995 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 14:54:13.658995 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:54:13.658995 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:54:13.658995 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:54:13.658995 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:54:13.658995 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:54:13.658995 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:54:13.658995 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:54:13.658995 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 26 14:54:13.658995 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:54:13.658995 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 26 14:54:13.658995 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:54:13.658995 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:54:13.660567 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:54:13.660567 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 26 14:54:13.660567 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:54:13.660567 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 26 14:54:13.660567 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 14:54:13.660567 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 14:54:13.660567 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 14:54:13.660567 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 14:54:13.660567 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 14:54:13.660567 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 14:54:13.660567 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 14:54:13.660567 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 26 14:54:13.660567 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 65/122] 10.222.0.46 () {56 vars in 1114 bytes} [Thu May 26 14:54:13 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:54:13.660567 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4579ef0-d702-4de7-b0e1-6e4c7fb7c5c8 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:13.675480 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:13.741651 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f9e948-c53d-4bb1-8446-c15a321d41e8 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:13.780332 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4579ef0-d702-4de7-b0e1-6e4c7fb7c5c8 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:13.790915 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8364fcc6-abf5-42c1-adb4-e19901d04cfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:13.791591 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8364fcc6-abf5-42c1-adb4-e19901d04cfe None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:13.791878 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8364fcc6-abf5-42c1-adb4-e19901d04cfe None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:13.792354 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8364fcc6-abf5-42c1-adb4-e19901d04cfe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:13.793213 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8364fcc6-abf5-42c1-adb4-e19901d04cfe None admin] RBAC: Adding query filter params (role.id=adb2011fe8f74ecbbb94465253e76c1c, scope.project.id=e474420e5c124a3f86383fce1ef4c4fb, user.id=990e3996e18d4a1aa532413cc2dbc3a1) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:54:13.793581 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8364fcc6-abf5-42c1-adb4-e19901d04cfe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:54:13.795052 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8364fcc6-abf5-42c1-adb4-e19901d04cfe None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:54:13.801152 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 58/123] 10.222.0.46 () {56 vars in 1374 bytes} [Thu May 26 14:54:13 2022] GET /identity/v3/role_assignments?user.id=990e3996e18d4a1aa532413cc2dbc3a1&scope.project.id=e474420e5c124a3f86383fce1ef4c4fb&role.id=adb2011fe8f74ecbbb94465253e76c1c => generated 259 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:54:13.802514 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adaebfd-7a76-478b-8009-751fee11c6ab None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:13.802803 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adaebfd-7a76-478b-8009-751fee11c6ab None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:13.803082 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adaebfd-7a76-478b-8009-751fee11c6ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:13.827680 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-0adaebfd-7a76-478b-8009-751fee11c6ab None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:54:13.829143 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-0adaebfd-7a76-478b-8009-751fee11c6ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:54:13.834970 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bed0541-373f-4128-96c5-999aac210d75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:13.836255 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bed0541-373f-4128-96c5-999aac210d75 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:13.836776 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bed0541-373f-4128-96c5-999aac210d75 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:13.837260 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bed0541-373f-4128-96c5-999aac210d75 None admin] PATH_INFO: `/v3/projects/e474420e5c124a3f86383fce1ef4c4fb` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:13.839004 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bed0541-373f-4128-96c5-999aac210d75 None admin] RBAC: Authorizing `identity:get_project(project_id=e474420e5c124a3f86383fce1ef4c4fb)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:54:13.841104 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bed0541-373f-4128-96c5-999aac210d75 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:54:13.843526 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 66/124] 10.222.0.46 () {56 vars in 1186 bytes} [Thu May 26 14:54:13 2022] GET /identity/v3/projects/e474420e5c124a3f86383fce1ef4c4fb => generated 298 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:54:13.844864 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bed0541-373f-4128-96c5-999aac210d75 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:13.890978 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:13.936591 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 59/125] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 14:54:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 14:54:13.943060 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adaebfd-7a76-478b-8009-751fee11c6ab None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:13.977538 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:14.027729 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bed0541-373f-4128-96c5-999aac210d75 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:14.076553 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b287b3a-22f9-4e14-b874-e08e7c54a820 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:14.077476 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b287b3a-22f9-4e14-b874-e08e7c54a820 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:14.077787 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b287b3a-22f9-4e14-b874-e08e7c54a820 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:14.078087 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b287b3a-22f9-4e14-b874-e08e7c54a820 None admin] PATH_INFO: `/v3/projects/e474420e5c124a3f86383fce1ef4c4fb` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:14.079484 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b287b3a-22f9-4e14-b874-e08e7c54a820 None admin] RBAC: Authorizing `identity:get_project(project_id=e474420e5c124a3f86383fce1ef4c4fb)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:54:14.081025 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b287b3a-22f9-4e14-b874-e08e7c54a820 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:54:14.083083 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 67/126] 10.222.0.46 () {56 vars in 1186 bytes} [Thu May 26 14:54:13 2022] GET /identity/v3/projects/e474420e5c124a3f86383fce1ef4c4fb => generated 298 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:54:14.083814 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b287b3a-22f9-4e14-b874-e08e7c54a820 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:14.125088 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adaebfd-7a76-478b-8009-751fee11c6ab None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:14.176889 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056a6bfc-5789-41f5-8844-d1ba85d042bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:14.178388 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056a6bfc-5789-41f5-8844-d1ba85d042bd None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:14.178876 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056a6bfc-5789-41f5-8844-d1ba85d042bd None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:14.179369 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056a6bfc-5789-41f5-8844-d1ba85d042bd None admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:14.181015 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056a6bfc-5789-41f5-8844-d1ba85d042bd None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:54:14.182265 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056a6bfc-5789-41f5-8844-d1ba85d042bd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:54:14.192375 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056a6bfc-5789-41f5-8844-d1ba85d042bd None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:54:14.201828 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 60/127] 10.222.0.46 () {56 vars in 1111 bytes} [Thu May 26 14:54:13 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:54:14.202955 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056a6bfc-5789-41f5-8844-d1ba85d042bd None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:14.209864 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b287b3a-22f9-4e14-b874-e08e7c54a820 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:14.241509 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:14.250083 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb32b61-79b9-4c5b-8338-7e0903829d2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:14.251049 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb32b61-79b9-4c5b-8338-7e0903829d2b None admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:14.251337 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb32b61-79b9-4c5b-8338-7e0903829d2b None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:14.251620 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb32b61-79b9-4c5b-8338-7e0903829d2b None admin] PATH_INFO: `/v3/projects/e474420e5c124a3f86383fce1ef4c4fb/users/70e4c37945d34ae18f6ac64fb237b230/roles/d4088d1845a24fbd9bd8975ce0b1d3d0` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:14.253908 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb32b61-79b9-4c5b-8338-7e0903829d2b None admin] RBAC: Authorizing `identity:create_grant(project_id=e474420e5c124a3f86383fce1ef4c4fb, user_id=70e4c37945d34ae18f6ac64fb237b230, role_id=d4088d1845a24fbd9bd8975ce0b1d3d0)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:54:14.255356 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb32b61-79b9-4c5b-8338-7e0903829d2b None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:54:14.281483 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 68/128] 10.222.0.46 () {58 vars in 1439 bytes} [Thu May 26 14:54:14 2022] PUT /identity/v3/projects/e474420e5c124a3f86383fce1ef4c4fb/users/70e4c37945d34ae18f6ac64fb237b230/roles/d4088d1845a24fbd9bd8975ce0b1d3d0 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 14:54:14.281483 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb32b61-79b9-4c5b-8338-7e0903829d2b None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:14.281729 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:14.366248 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056a6bfc-5789-41f5-8844-d1ba85d042bd None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:14.417282 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb32b61-79b9-4c5b-8338-7e0903829d2b None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:14.467462 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2def61-1d32-48e1-82d3-191d0047fd60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:14.467795 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2d10b1-c033-4533-babe-123b61cf9fd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:14.468758 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2d10b1-c033-4533-babe-123b61cf9fd4 None admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:14.469092 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2d10b1-c033-4533-babe-123b61cf9fd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:14.469374 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2d10b1-c033-4533-babe-123b61cf9fd4 None admin] PATH_INFO: `/v3/projects/e474420e5c124a3f86383fce1ef4c4fb/users/dede0ef3af5b453796bce68732b5fd29/roles/a2e551773cbd4df084020676865b8222` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:14.469629 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2def61-1d32-48e1-82d3-191d0047fd60 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:14.470107 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2def61-1d32-48e1-82d3-191d0047fd60 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:14.470616 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2def61-1d32-48e1-82d3-191d0047fd60 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:14.477482 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2d10b1-c033-4533-babe-123b61cf9fd4 None admin] RBAC: Authorizing `identity:create_grant(project_id=e474420e5c124a3f86383fce1ef4c4fb, user_id=dede0ef3af5b453796bce68732b5fd29, role_id=a2e551773cbd4df084020676865b8222)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:54:14.477482 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2d10b1-c033-4533-babe-123b61cf9fd4 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:54:14.494341 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 69/129] 10.222.0.46 () {58 vars in 1439 bytes} [Thu May 26 14:54:14 2022] PUT /identity/v3/projects/e474420e5c124a3f86383fce1ef4c4fb/users/dede0ef3af5b453796bce68732b5fd29/roles/a2e551773cbd4df084020676865b8222 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 14:54:14.495870 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2d10b1-c033-4533-babe-123b61cf9fd4 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:14.504977 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2def61-1d32-48e1-82d3-191d0047fd60 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 14:54:14.524595 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-1c2def61-1d32-48e1-82d3-191d0047fd60 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 26 14:54:14.524595 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:54:14.524595 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:54:14.524595 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:54:14.524595 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:54:14.524595 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:54:14.524595 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:54:14.524595 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:54:14.524595 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:54:14.524595 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:54:14.524595 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:54:14.524595 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:54:14.524595 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 14:54:14.524595 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 14:54:14.524595 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 14:54:14.524595 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 14:54:14.525554 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 14:54:14.525554 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 14:54:14.525554 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 14:54:14.525554 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 14:54:14.525554 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:54:14.525554 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:54:14.525554 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:54:14.525554 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:54:14.525554 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:54:14.525554 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:54:14.525554 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:54:14.525554 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 26 14:54:14.525554 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:54:14.525554 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 26 14:54:14.525554 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:54:14.525554 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:54:14.526322 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:54:14.526322 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 26 14:54:14.526322 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:54:14.526322 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 26 14:54:14.526322 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 14:54:14.526322 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 14:54:14.526322 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 14:54:14.526322 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 14:54:14.526322 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 14:54:14.526322 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 14:54:14.526322 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 26 14:54:14.526322 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 26 14:54:14.530913 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 61/130] 10.222.0.46 () {56 vars in 1099 bytes} [Thu May 26 14:54:14 2022] GET /identity/v3/roles/member => generated 84 bytes in 328 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:54:14.537070 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2def61-1d32-48e1-82d3-191d0047fd60 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:14.662276 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:14.723478 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:14.734743 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2def61-1d32-48e1-82d3-191d0047fd60 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:14.776525 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5427baf-4daf-4a32-bc19-e3d07b9227b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:14.777237 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5427baf-4daf-4a32-bc19-e3d07b9227b9 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:14.777556 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5427baf-4daf-4a32-bc19-e3d07b9227b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:14.777876 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5427baf-4daf-4a32-bc19-e3d07b9227b9 None admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:14.778721 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5427baf-4daf-4a32-bc19-e3d07b9227b9 None admin] RBAC: Adding query filter params (name=member) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:54:14.779313 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5427baf-4daf-4a32-bc19-e3d07b9227b9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:54:14.780873 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5427baf-4daf-4a32-bc19-e3d07b9227b9 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:54:14.790427 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 62/131] 10.222.0.46 () {56 vars in 1101 bytes} [Thu May 26 14:54:14 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:54:14.796298 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5427baf-4daf-4a32-bc19-e3d07b9227b9 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:14.801444 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2d10b1-c033-4533-babe-123b61cf9fd4 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:14.869388 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027de3d9-01ee-4bc8-93b8-c3dacf6d5c85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:14.870474 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027de3d9-01ee-4bc8-93b8-c3dacf6d5c85 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:14.870828 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027de3d9-01ee-4bc8-93b8-c3dacf6d5c85 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:14.871144 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027de3d9-01ee-4bc8-93b8-c3dacf6d5c85 None admin] PATH_INFO: `/v3/users/70e4c37945d34ae18f6ac64fb237b230` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:14.873127 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027de3d9-01ee-4bc8-93b8-c3dacf6d5c85 None admin] RBAC: Authorizing `identity:get_user(user_id=70e4c37945d34ae18f6ac64fb237b230)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:54:14.874758 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027de3d9-01ee-4bc8-93b8-c3dacf6d5c85 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:54:14.876765 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 70/132] 10.222.0.46 () {56 vars in 1177 bytes} [Thu May 26 14:54:14 2022] GET /identity/v3/users/70e4c37945d34ae18f6ac64fb237b230 => generated 275 bytes in 382 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:54:14.882380 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027de3d9-01ee-4bc8-93b8-c3dacf6d5c85 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:14.910234 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5427baf-4daf-4a32-bc19-e3d07b9227b9 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:14.950451 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86434ee3-ab54-4fa2-a930-b7de54c6738f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:14.951361 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86434ee3-ab54-4fa2-a930-b7de54c6738f None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:14.951650 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86434ee3-ab54-4fa2-a930-b7de54c6738f None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:14.951931 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86434ee3-ab54-4fa2-a930-b7de54c6738f None admin] PATH_INFO: `/v3/users/70e4c37945d34ae18f6ac64fb237b230` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:14.953183 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86434ee3-ab54-4fa2-a930-b7de54c6738f None admin] RBAC: Authorizing `identity:get_user(user_id=70e4c37945d34ae18f6ac64fb237b230)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:54:14.954889 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86434ee3-ab54-4fa2-a930-b7de54c6738f None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:54:14.961487 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 63/133] 10.222.0.46 () {56 vars in 1177 bytes} [Thu May 26 14:54:14 2022] GET /identity/v3/users/70e4c37945d34ae18f6ac64fb237b230 => generated 275 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:54:14.962894 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86434ee3-ab54-4fa2-a930-b7de54c6738f None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:14.997944 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:15.078515 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86434ee3-ab54-4fa2-a930-b7de54c6738f None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:15.116309 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb82fbf-1181-4c5f-b143-082afb810d25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:15.117208 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb82fbf-1181-4c5f-b143-082afb810d25 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:15.117572 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb82fbf-1181-4c5f-b143-082afb810d25 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:15.117873 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb82fbf-1181-4c5f-b143-082afb810d25 None admin] PATH_INFO: `/v3/projects/1c95d1c3fe20488caf82b4b1125bfbc1` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:15.123562 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb82fbf-1181-4c5f-b143-082afb810d25 None admin] RBAC: Authorizing `identity:get_project(project_id=1c95d1c3fe20488caf82b4b1125bfbc1)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:54:15.125268 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb82fbf-1181-4c5f-b143-082afb810d25 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:54:15.139036 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 64/134] 10.222.0.46 () {56 vars in 1186 bytes} [Thu May 26 14:54:14 2022] GET /identity/v3/projects/1c95d1c3fe20488caf82b4b1125bfbc1 => generated 312 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 14:54:15.140111 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb82fbf-1181-4c5f-b143-082afb810d25 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:15.202878 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027de3d9-01ee-4bc8-93b8-c3dacf6d5c85 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:15.266460 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:15.279850 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f26589-3e6c-4d78-8bd7-e55c8dc55d0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:15.280742 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f26589-3e6c-4d78-8bd7-e55c8dc55d0b None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:15.281027 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f26589-3e6c-4d78-8bd7-e55c8dc55d0b None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:15.281308 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47f26589-3e6c-4d78-8bd7-e55c8dc55d0b None admin] PATH_INFO: `/v3/projects/e474420e5c124a3f86383fce1ef4c4fb` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:15.283605 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f26589-3e6c-4d78-8bd7-e55c8dc55d0b None admin] RBAC: Authorizing `identity:get_project(project_id=e474420e5c124a3f86383fce1ef4c4fb)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:54:15.285374 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47f26589-3e6c-4d78-8bd7-e55c8dc55d0b None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:54:15.287286 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 71/135] 10.222.0.46 () {56 vars in 1186 bytes} [Thu May 26 14:54:14 2022] GET /identity/v3/projects/e474420e5c124a3f86383fce1ef4c4fb => generated 298 bytes in 405 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:54:15.296637 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f26589-3e6c-4d78-8bd7-e55c8dc55d0b None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:15.381483 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:15.399434 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb82fbf-1181-4c5f-b143-082afb810d25 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:15.482833 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebeb98b-38ec-44b9-bbf8-fd59507ec946 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:15.483762 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebeb98b-38ec-44b9-bbf8-fd59507ec946 None admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:15.484052 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebeb98b-38ec-44b9-bbf8-fd59507ec946 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:15.484340 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebeb98b-38ec-44b9-bbf8-fd59507ec946 None admin] PATH_INFO: `/v3/projects/1c95d1c3fe20488caf82b4b1125bfbc1/users/70e4c37945d34ae18f6ac64fb237b230/roles/d4088d1845a24fbd9bd8975ce0b1d3d0` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:15.486440 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebeb98b-38ec-44b9-bbf8-fd59507ec946 None admin] RBAC: Authorizing `identity:create_grant(project_id=1c95d1c3fe20488caf82b4b1125bfbc1, user_id=70e4c37945d34ae18f6ac64fb237b230, role_id=d4088d1845a24fbd9bd8975ce0b1d3d0)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:54:15.487933 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ebeb98b-38ec-44b9-bbf8-fd59507ec946 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:54:15.508941 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 65/136] 10.222.0.46 () {58 vars in 1439 bytes} [Thu May 26 14:54:15 2022] PUT /identity/v3/projects/1c95d1c3fe20488caf82b4b1125bfbc1/users/70e4c37945d34ae18f6ac64fb237b230/roles/d4088d1845a24fbd9bd8975ce0b1d3d0 => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 14:54:15.510841 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d340711c-22b0-459c-9c08-5776d83f5965 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:15.511142 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d340711c-22b0-459c-9c08-5776d83f5965 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:15.511485 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d340711c-22b0-459c-9c08-5776d83f5965 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:15.513429 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 66/137] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 14:54:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 14:54:15.519012 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac434369-ca23-49da-b063-9d1c14c3aeaa None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:15.519266 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac434369-ca23-49da-b063-9d1c14c3aeaa None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:15.519515 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac434369-ca23-49da-b063-9d1c14c3aeaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:15.528143 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47f26589-3e6c-4d78-8bd7-e55c8dc55d0b None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:15.554567 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-ac434369-ca23-49da-b063-9d1c14c3aeaa None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:54:15.555878 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-ac434369-ca23-49da-b063-9d1c14c3aeaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:54:15.616313 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa3f3d5-37d6-4723-92d6-9ec4b55e8af2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:15.617666 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa3f3d5-37d6-4723-92d6-9ec4b55e8af2 None admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:15.617965 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa3f3d5-37d6-4723-92d6-9ec4b55e8af2 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:15.618256 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa3f3d5-37d6-4723-92d6-9ec4b55e8af2 None admin] PATH_INFO: `/v3/projects/e474420e5c124a3f86383fce1ef4c4fb/users/70e4c37945d34ae18f6ac64fb237b230/roles/160115b2143c4b389d39cd93f43e1d01` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:15.624253 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa3f3d5-37d6-4723-92d6-9ec4b55e8af2 None admin] RBAC: Authorizing `identity:create_grant(project_id=e474420e5c124a3f86383fce1ef4c4fb, user_id=70e4c37945d34ae18f6ac64fb237b230, role_id=160115b2143c4b389d39cd93f43e1d01)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:54:15.627895 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa3f3d5-37d6-4723-92d6-9ec4b55e8af2 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:54:15.669520 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 72/138] 10.222.0.46 () {58 vars in 1439 bytes} [Thu May 26 14:54:15 2022] PUT /identity/v3/projects/e474420e5c124a3f86383fce1ef4c4fb/users/70e4c37945d34ae18f6ac64fb237b230/roles/160115b2143c4b389d39cd93f43e1d01 => generated 0 bytes in 377 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 14:54:15.720098 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 67/139] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 14:54:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 14:54:15.763705 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa3f3d5-37d6-4723-92d6-9ec4b55e8af2 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:15.848202 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:15.905072 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa3f3d5-37d6-4723-92d6-9ec4b55e8af2 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:15.953400 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad5c694-52d4-4240-8a23-40200fff50c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:15.954324 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad5c694-52d4-4240-8a23-40200fff50c1 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:15.954668 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad5c694-52d4-4240-8a23-40200fff50c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:15.954969 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad5c694-52d4-4240-8a23-40200fff50c1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:15.956204 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad5c694-52d4-4240-8a23-40200fff50c1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:54:15.957917 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad5c694-52d4-4240-8a23-40200fff50c1 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:54:15.959676 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 73/140] 10.222.0.46 () {56 vars in 1108 bytes} [Thu May 26 14:54:15 2022] GET /identity/v3/domains/default => generated 199 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:54:15.966177 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac434369-ca23-49da-b063-9d1c14c3aeaa None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:16.114165 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:16.135825 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac434369-ca23-49da-b063-9d1c14c3aeaa None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:16.181892 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4b4db2-14f8-4ddb-96da-f5510011f7fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:16.182582 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4b4db2-14f8-4ddb-96da-f5510011f7fa None admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:16.182870 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4b4db2-14f8-4ddb-96da-f5510011f7fa None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:16.183152 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4b4db2-14f8-4ddb-96da-f5510011f7fa None admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:16.184410 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4b4db2-14f8-4ddb-96da-f5510011f7fa None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:54:16.185928 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4b4db2-14f8-4ddb-96da-f5510011f7fa None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:54:16.218213 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 68/141] 10.222.0.46 () {60 vars in 1132 bytes} [Thu May 26 14:54:15 2022] POST /identity/v3/users => generated 283 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 14:54:16.763460 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b13c46-8cbe-4d4d-99fd-f0771a4beed1 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:16.764122 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b13c46-8cbe-4d4d-99fd-f0771a4beed1 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:16.764122 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b13c46-8cbe-4d4d-99fd-f0771a4beed1 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:16.765368 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 74/142] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 14:54:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 14:54:16.771917 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e203bd8b-5d88-4360-aa5d-a46478b5c701 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:16.772715 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e203bd8b-5d88-4360-aa5d-a46478b5c701 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:16.773141 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e203bd8b-5d88-4360-aa5d-a46478b5c701 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:16.799475 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-e203bd8b-5d88-4360-aa5d-a46478b5c701 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:54:16.800921 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-e203bd8b-5d88-4360-aa5d-a46478b5c701 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:54:16.885935 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 69/143] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 14:54:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 14:54:16.921470 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b13c46-8cbe-4d4d-99fd-f0771a4beed1 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:17.076068 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b13c46-8cbe-4d4d-99fd-f0771a4beed1 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:17.112640 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:17.122425 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761ed853-d9b4-4cac-abe1-6d3a9c6318da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:17.123302 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761ed853-d9b4-4cac-abe1-6d3a9c6318da None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:17.123813 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761ed853-d9b4-4cac-abe1-6d3a9c6318da None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:17.124138 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761ed853-d9b4-4cac-abe1-6d3a9c6318da None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:17.145855 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761ed853-d9b4-4cac-abe1-6d3a9c6318da None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 14:54:17.154271 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-761ed853-d9b4-4cac-abe1-6d3a9c6318da None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 26 14:54:17.154271 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:54:17.154271 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:54:17.154271 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:54:17.154271 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:54:17.154271 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:54:17.154271 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:54:17.154271 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:54:17.154271 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:54:17.154271 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:54:17.154271 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:54:17.154271 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:54:17.154271 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 14:54:17.154271 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 14:54:17.154271 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 14:54:17.154271 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 14:54:17.155225 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 14:54:17.155225 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 14:54:17.155225 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 14:54:17.155225 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 14:54:17.155225 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:54:17.155225 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:54:17.155225 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:54:17.155225 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:54:17.155225 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:54:17.155225 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:54:17.155225 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:54:17.155225 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 26 14:54:17.155225 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:54:17.155225 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 26 14:54:17.155225 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:54:17.155225 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:54:17.155988 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:54:17.155988 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 26 14:54:17.155988 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:54:17.155988 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 26 14:54:17.155988 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 14:54:17.155988 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 14:54:17.155988 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 14:54:17.155988 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 14:54:17.155988 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 14:54:17.155988 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 14:54:17.155988 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 26 14:54:17.155988 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 26 14:54:17.155988 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 75/144] 10.222.0.46 () {56 vars in 1099 bytes} [Thu May 26 14:54:16 2022] GET /identity/v3/roles/reader => generated 84 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:54:17.162688 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e203bd8b-5d88-4360-aa5d-a46478b5c701 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:17.279069 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:17.314731 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e203bd8b-5d88-4360-aa5d-a46478b5c701 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:17.364618 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28b2e05-1436-4202-86db-557ca6b03c0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:17.365363 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28b2e05-1436-4202-86db-557ca6b03c0b None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:17.365711 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28b2e05-1436-4202-86db-557ca6b03c0b None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:17.365951 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28b2e05-1436-4202-86db-557ca6b03c0b None admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:17.366720 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28b2e05-1436-4202-86db-557ca6b03c0b None admin] RBAC: Adding query filter params (name=reader) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:54:17.367031 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28b2e05-1436-4202-86db-557ca6b03c0b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:54:17.368450 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28b2e05-1436-4202-86db-557ca6b03c0b None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:54:17.381794 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 70/145] 10.222.0.46 () {56 vars in 1101 bytes} [Thu May 26 14:54:17 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:54:17.389356 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761ed853-d9b4-4cac-abe1-6d3a9c6318da None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:17.523602 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761ed853-d9b4-4cac-abe1-6d3a9c6318da None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:17.567924 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d6acdb-d7e6-4566-b5b6-7982de6be7c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:17.568832 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d6acdb-d7e6-4566-b5b6-7982de6be7c5 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:17.569115 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d6acdb-d7e6-4566-b5b6-7982de6be7c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:17.569390 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d6acdb-d7e6-4566-b5b6-7982de6be7c5 None admin] PATH_INFO: `/v3/users/990e3996e18d4a1aa532413cc2dbc3a1` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:17.570656 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d6acdb-d7e6-4566-b5b6-7982de6be7c5 None admin] RBAC: Authorizing `identity:get_user(user_id=990e3996e18d4a1aa532413cc2dbc3a1)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:54:17.572129 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d6acdb-d7e6-4566-b5b6-7982de6be7c5 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:54:17.573801 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 76/146] 10.222.0.46 () {56 vars in 1177 bytes} [Thu May 26 14:54:17 2022] GET /identity/v3/users/990e3996e18d4a1aa532413cc2dbc3a1 => generated 289 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:54:17.580594 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28b2e05-1436-4202-86db-557ca6b03c0b None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:17.764323 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28b2e05-1436-4202-86db-557ca6b03c0b None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:17.806962 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:17.815314 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4c5fe6-fc18-4ce4-ab9e-06b57e12a3d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:17.816705 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4c5fe6-fc18-4ce4-ab9e-06b57e12a3d3 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:17.817439 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4c5fe6-fc18-4ce4-ab9e-06b57e12a3d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:17.817759 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4c5fe6-fc18-4ce4-ab9e-06b57e12a3d3 None admin] PATH_INFO: `/v3/projects/e474420e5c124a3f86383fce1ef4c4fb` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:17.822296 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4c5fe6-fc18-4ce4-ab9e-06b57e12a3d3 None admin] RBAC: Authorizing `identity:get_project(project_id=e474420e5c124a3f86383fce1ef4c4fb)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:54:17.823788 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4c5fe6-fc18-4ce4-ab9e-06b57e12a3d3 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:54:17.826205 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 71/147] 10.222.0.46 () {56 vars in 1186 bytes} [Thu May 26 14:54:17 2022] GET /identity/v3/projects/e474420e5c124a3f86383fce1ef4c4fb => generated 298 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:54:17.835165 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d6acdb-d7e6-4566-b5b6-7982de6be7c5 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:17.977214 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:18.062196 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d6acdb-d7e6-4566-b5b6-7982de6be7c5 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:18.123955 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393d1012-b072-4d43-b78a-42d61ea62e2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:18.125588 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393d1012-b072-4d43-b78a-42d61ea62e2a None admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:18.126087 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393d1012-b072-4d43-b78a-42d61ea62e2a None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:18.126584 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393d1012-b072-4d43-b78a-42d61ea62e2a None admin] PATH_INFO: `/v3/projects/e474420e5c124a3f86383fce1ef4c4fb/users/990e3996e18d4a1aa532413cc2dbc3a1/roles/adb2011fe8f74ecbbb94465253e76c1c` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:18.130253 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393d1012-b072-4d43-b78a-42d61ea62e2a None admin] RBAC: Authorizing `identity:create_grant(project_id=e474420e5c124a3f86383fce1ef4c4fb, user_id=990e3996e18d4a1aa532413cc2dbc3a1, role_id=adb2011fe8f74ecbbb94465253e76c1c)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:54:18.133235 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393d1012-b072-4d43-b78a-42d61ea62e2a None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:54:18.165470 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 77/148] 10.222.0.46 () {58 vars in 1439 bytes} [Thu May 26 14:54:17 2022] PUT /identity/v3/projects/e474420e5c124a3f86383fce1ef4c4fb/users/990e3996e18d4a1aa532413cc2dbc3a1/roles/adb2011fe8f74ecbbb94465253e76c1c => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 14:54:18.279368 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98b0792-7585-4ec1-a69b-ce0aa77cfac5 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:18.280157 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98b0792-7585-4ec1-a69b-ce0aa77cfac5 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:18.280453 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98b0792-7585-4ec1-a69b-ce0aa77cfac5 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:18.282215 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 72/149] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 14:54:18 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 14:54:18.288638 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b03e810-6464-4d41-b70c-3c4d14dc8f32 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:18.289138 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b03e810-6464-4d41-b70c-3c4d14dc8f32 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:18.289688 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b03e810-6464-4d41-b70c-3c4d14dc8f32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:18.328020 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-3b03e810-6464-4d41-b70c-3c4d14dc8f32 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:54:18.329977 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-3b03e810-6464-4d41-b70c-3c4d14dc8f32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:54:18.363488 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c06907-ebe1-432b-85a6-403613540e53 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:18.364140 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c06907-ebe1-432b-85a6-403613540e53 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:18.364645 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c06907-ebe1-432b-85a6-403613540e53 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:18.366287 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 73/150] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 14:54:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 14:54:18.374022 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5f6967-b3f0-45f8-95f0-732c7bc23724 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:18.374624 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5f6967-b3f0-45f8-95f0-732c7bc23724 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:18.375109 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5f6967-b3f0-45f8-95f0-732c7bc23724 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:18.406949 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-7a5f6967-b3f0-45f8-95f0-732c7bc23724 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:54:18.408537 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-7a5f6967-b3f0-45f8-95f0-732c7bc23724 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:54:18.508500 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 78/151] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 14:54:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 14:54:18.554397 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b03e810-6464-4d41-b70c-3c4d14dc8f32 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:18.558834 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 74/152] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 14:54:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 14:54:18.660232 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5f6967-b3f0-45f8-95f0-732c7bc23724 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:18.673541 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:18.805530 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b03e810-6464-4d41-b70c-3c4d14dc8f32 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:18.874137 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47fc036-520e-4222-9073-7dfe9d996710 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:18.875214 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47fc036-520e-4222-9073-7dfe9d996710 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:18.875622 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47fc036-520e-4222-9073-7dfe9d996710 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:18.876021 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47fc036-520e-4222-9073-7dfe9d996710 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:18.892967 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47fc036-520e-4222-9073-7dfe9d996710 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 14:54:18.912167 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-b47fc036-520e-4222-9073-7dfe9d996710 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 14:54:18.912167 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:54:18.912167 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:54:18.912167 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:54:18.912167 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:54:18.912167 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:54:18.912167 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:54:18.912167 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:54:18.912167 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:54:18.912167 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:54:18.912167 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:54:18.912167 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:54:18.912167 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 14:54:18.912167 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 14:54:18.912167 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 14:54:18.912167 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 14:54:18.913097 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 14:54:18.913097 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 14:54:18.913097 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 14:54:18.913097 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 14:54:18.913097 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:54:18.913097 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:54:18.913097 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:54:18.913097 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:54:18.913097 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:54:18.913097 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:54:18.913097 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:54:18.913097 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 26 14:54:18.913097 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:54:18.913097 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 26 14:54:18.913097 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:54:18.913097 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:54:18.913878 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:54:18.913878 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 26 14:54:18.913878 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:54:18.913878 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 26 14:54:18.913878 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 14:54:18.913878 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 14:54:18.913878 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 14:54:18.913878 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 14:54:18.913878 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 14:54:18.913878 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 14:54:18.913878 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 14:54:18.913878 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 26 14:54:18.914444 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5f6967-b3f0-45f8-95f0-732c7bc23724 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:18.914846 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 79/153] 10.222.0.46 () {56 vars in 1096 bytes} [Thu May 26 14:54:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 361 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:54:18.922771 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47fc036-520e-4222-9073-7dfe9d996710 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:18.978480 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:18.990967 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67758ff-20db-49ce-b25a-7602b1c73dc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:18.991784 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67758ff-20db-49ce-b25a-7602b1c73dc5 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:18.992069 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67758ff-20db-49ce-b25a-7602b1c73dc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:18.992347 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67758ff-20db-49ce-b25a-7602b1c73dc5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:19.014081 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67758ff-20db-49ce-b25a-7602b1c73dc5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 14:54:19.022298 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-d67758ff-20db-49ce-b25a-7602b1c73dc5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 26 14:54:19.022298 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:54:19.022298 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:54:19.022298 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:54:19.022298 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:54:19.022298 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:54:19.022298 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:54:19.022298 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:54:19.022298 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:54:19.022298 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:54:19.022298 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:54:19.022298 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:54:19.022298 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 14:54:19.022298 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 14:54:19.022298 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 14:54:19.022298 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 14:54:19.023194 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 14:54:19.023194 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 14:54:19.023194 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 14:54:19.023194 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 14:54:19.023194 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:54:19.023194 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:54:19.023194 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:54:19.023194 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:54:19.023194 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:54:19.023194 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:54:19.023194 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:54:19.023194 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 26 14:54:19.023194 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:54:19.023194 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 26 14:54:19.023194 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:54:19.023194 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:54:19.024144 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:54:19.024144 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 26 14:54:19.024144 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:54:19.024144 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 26 14:54:19.024144 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 14:54:19.024144 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 14:54:19.024144 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 14:54:19.024144 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 14:54:19.024144 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 14:54:19.024144 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 14:54:19.024144 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 26 14:54:19.024144 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 26 14:54:19.026532 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 75/154] 10.222.0.46 () {56 vars in 1099 bytes} [Thu May 26 14:54:18 2022] GET /identity/v3/roles/member => generated 84 bytes in 364 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:54:19.029320 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67758ff-20db-49ce-b25a-7602b1c73dc5 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:19.109112 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47fc036-520e-4222-9073-7dfe9d996710 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:19.126138 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:19.184642 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e5e122-c99d-4e34-87db-f68a9488ba9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:19.185722 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e5e122-c99d-4e34-87db-f68a9488ba9a None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:19.186198 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e5e122-c99d-4e34-87db-f68a9488ba9a None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:19.186670 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e5e122-c99d-4e34-87db-f68a9488ba9a None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:19.187635 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e5e122-c99d-4e34-87db-f68a9488ba9a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:54:19.188145 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e5e122-c99d-4e34-87db-f68a9488ba9a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:54:19.189758 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e5e122-c99d-4e34-87db-f68a9488ba9a None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:54:19.208261 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 80/155] 10.222.0.46 () {56 vars in 1099 bytes} [Thu May 26 14:54:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:54:19.216425 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e5e122-c99d-4e34-87db-f68a9488ba9a None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:19.256890 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67758ff-20db-49ce-b25a-7602b1c73dc5 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:19.325752 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61e3204-cceb-4a84-84d4-d4e72a7ddea1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:19.326577 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61e3204-cceb-4a84-84d4-d4e72a7ddea1 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:19.326873 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61e3204-cceb-4a84-84d4-d4e72a7ddea1 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:19.327157 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61e3204-cceb-4a84-84d4-d4e72a7ddea1 None admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:19.327932 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61e3204-cceb-4a84-84d4-d4e72a7ddea1 None admin] RBAC: Adding query filter params (name=member) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:54:19.328241 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61e3204-cceb-4a84-84d4-d4e72a7ddea1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:54:19.333849 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61e3204-cceb-4a84-84d4-d4e72a7ddea1 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:54:19.349478 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 76/156] 10.222.0.46 () {56 vars in 1101 bytes} [Thu May 26 14:54:19 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:54:19.354603 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61e3204-cceb-4a84-84d4-d4e72a7ddea1 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:19.425527 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:19.437317 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e5e122-c99d-4e34-87db-f68a9488ba9a None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:19.498236 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61e3204-cceb-4a84-84d4-d4e72a7ddea1 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:19.520852 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4e00fc-6196-483c-b66a-11dbaa6997cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:19.522920 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4e00fc-6196-483c-b66a-11dbaa6997cb None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:19.523697 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4e00fc-6196-483c-b66a-11dbaa6997cb None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:19.529488 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4e00fc-6196-483c-b66a-11dbaa6997cb None admin] PATH_INFO: `/v3/users/dede0ef3af5b453796bce68732b5fd29` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:19.532207 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4e00fc-6196-483c-b66a-11dbaa6997cb None admin] RBAC: Authorizing `identity:get_user(user_id=dede0ef3af5b453796bce68732b5fd29)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:54:19.535019 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4e00fc-6196-483c-b66a-11dbaa6997cb None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:54:19.538324 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 81/157] 10.222.0.46 () {56 vars in 1177 bytes} [Thu May 26 14:54:19 2022] GET /identity/v3/users/dede0ef3af5b453796bce68732b5fd29 => generated 247 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:54:19.546839 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4e00fc-6196-483c-b66a-11dbaa6997cb None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:19.554127 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7347ef80-38ab-4040-8e6c-0d66c325f5d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:19.554127 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7347ef80-38ab-4040-8e6c-0d66c325f5d5 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:19.554484 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7347ef80-38ab-4040-8e6c-0d66c325f5d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:19.554588 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7347ef80-38ab-4040-8e6c-0d66c325f5d5 None admin] PATH_INFO: `/v3/users/70e4c37945d34ae18f6ac64fb237b230` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:19.555862 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7347ef80-38ab-4040-8e6c-0d66c325f5d5 None admin] RBAC: Authorizing `identity:get_user(user_id=70e4c37945d34ae18f6ac64fb237b230)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:54:19.561603 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7347ef80-38ab-4040-8e6c-0d66c325f5d5 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:54:19.561603 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 77/158] 10.222.0.46 () {56 vars in 1177 bytes} [Thu May 26 14:54:19 2022] GET /identity/v3/users/70e4c37945d34ae18f6ac64fb237b230 => generated 275 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:54:19.565656 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7347ef80-38ab-4040-8e6c-0d66c325f5d5 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:19.623652 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:19.752418 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4e00fc-6196-483c-b66a-11dbaa6997cb None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:19.775321 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7347ef80-38ab-4040-8e6c-0d66c325f5d5 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:19.837861 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:19.855802 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8104993c-60d1-497c-a2c4-57859813a2a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:19.856722 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8104993c-60d1-497c-a2c4-57859813a2a5 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:19.857123 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8104993c-60d1-497c-a2c4-57859813a2a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:19.857542 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8104993c-60d1-497c-a2c4-57859813a2a5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:19.890446 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9a8715-5182-465c-93ba-cb9b8a1ccddc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:19.891122 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9a8715-5182-465c-93ba-cb9b8a1ccddc None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:19.891410 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9a8715-5182-465c-93ba-cb9b8a1ccddc None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:19.891696 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9a8715-5182-465c-93ba-cb9b8a1ccddc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:20.015482 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8104993c-60d1-497c-a2c4-57859813a2a5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:54:20.017478 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8104993c-60d1-497c-a2c4-57859813a2a5 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:54:20.078304 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9a8715-5182-465c-93ba-cb9b8a1ccddc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:54:20.079888 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9a8715-5182-465c-93ba-cb9b8a1ccddc None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:54:20.113487 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:20.240500 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 82/159] 10.222.0.46 () {58 vars in 1282 bytes} [Thu May 26 14:54:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 695 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 14:54:20.242266 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b45bb6-58b2-4014-8657-5d63f28926c9 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:20.242777 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b45bb6-58b2-4014-8657-5d63f28926c9 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:20.243245 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b45bb6-58b2-4014-8657-5d63f28926c9 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:20.245093 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 83/160] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 14:54:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 14:54:20.250660 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6873d39-f981-4fcb-bc92-1a2131690e16 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:20.251215 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6873d39-f981-4fcb-bc92-1a2131690e16 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:20.251697 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6873d39-f981-4fcb-bc92-1a2131690e16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:20.270405 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-d6873d39-f981-4fcb-bc92-1a2131690e16 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:54:20.272094 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-d6873d39-f981-4fcb-bc92-1a2131690e16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:54:20.342246 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 78/161] 10.222.0.46 () {58 vars in 1282 bytes} [Thu May 26 14:54:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 778 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 26 14:54:20.343109 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9a8715-5182-465c-93ba-cb9b8a1ccddc None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:20.384383 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:20.455166 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 84/162] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 14:54:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 14:54:20.456065 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6873d39-f981-4fcb-bc92-1a2131690e16 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:20.516659 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9a8715-5182-465c-93ba-cb9b8a1ccddc None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:20.595418 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6362c80f-8383-47ef-8a8b-738ec103b46d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:20.596319 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6362c80f-8383-47ef-8a8b-738ec103b46d None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:20.596603 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6362c80f-8383-47ef-8a8b-738ec103b46d None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:20.596884 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6362c80f-8383-47ef-8a8b-738ec103b46d None admin] PATH_INFO: `/v3/projects/e474420e5c124a3f86383fce1ef4c4fb` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:20.598145 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6362c80f-8383-47ef-8a8b-738ec103b46d None admin] RBAC: Authorizing `identity:get_project(project_id=e474420e5c124a3f86383fce1ef4c4fb)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:54:20.600076 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6362c80f-8383-47ef-8a8b-738ec103b46d None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:54:20.602367 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 79/163] 10.222.0.46 () {56 vars in 1186 bytes} [Thu May 26 14:54:20 2022] GET /identity/v3/projects/e474420e5c124a3f86383fce1ef4c4fb => generated 298 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:54:20.603155 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6362c80f-8383-47ef-8a8b-738ec103b46d None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:20.648439 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:20.684276 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6873d39-f981-4fcb-bc92-1a2131690e16 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:20.731943 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c871ccca-d526-46a7-bac8-2a5fd8b8b5f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:20.732861 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c871ccca-d526-46a7-bac8-2a5fd8b8b5f8 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:20.733150 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c871ccca-d526-46a7-bac8-2a5fd8b8b5f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:20.733422 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c871ccca-d526-46a7-bac8-2a5fd8b8b5f8 None admin] PATH_INFO: `/v3/projects/e474420e5c124a3f86383fce1ef4c4fb` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:20.738999 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c871ccca-d526-46a7-bac8-2a5fd8b8b5f8 None admin] RBAC: Authorizing `identity:get_project(project_id=e474420e5c124a3f86383fce1ef4c4fb)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:54:20.740482 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c871ccca-d526-46a7-bac8-2a5fd8b8b5f8 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:54:20.742252 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 85/164] 10.222.0.46 () {56 vars in 1186 bytes} [Thu May 26 14:54:20 2022] GET /identity/v3/projects/e474420e5c124a3f86383fce1ef4c4fb => generated 298 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:54:20.743480 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a2c5f9-deb8-4960-86ce-cc789eb51156 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:20.743764 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a2c5f9-deb8-4960-86ce-cc789eb51156 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:20.744038 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a2c5f9-deb8-4960-86ce-cc789eb51156 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:20.745267 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 86/165] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 14:54:20 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 14:54:20.745991 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a2c5f9-deb8-4960-86ce-cc789eb51156 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:20.860605 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:20.888331 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6362c80f-8383-47ef-8a8b-738ec103b46d None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:20.951713 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc083aa-fab2-4d5d-a56a-166924569785 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:20.952544 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc083aa-fab2-4d5d-a56a-166924569785 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:20.952836 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc083aa-fab2-4d5d-a56a-166924569785 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:20.953133 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc083aa-fab2-4d5d-a56a-166924569785 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:20.975410 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc083aa-fab2-4d5d-a56a-166924569785 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 14:54:20.984678 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-4bc083aa-fab2-4d5d-a56a-166924569785 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 26 14:54:20.984678 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:54:20.984678 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:54:20.984678 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:54:20.984678 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:54:20.984678 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:54:20.984678 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:54:20.984678 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:54:20.984678 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:54:20.984678 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:54:20.984678 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:54:20.984678 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:54:20.984678 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 14:54:20.984678 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 14:54:20.984678 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 14:54:20.984678 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 14:54:20.985643 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 14:54:20.985643 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 14:54:20.985643 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 14:54:20.985643 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 14:54:20.985643 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:54:20.985643 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:54:20.985643 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:54:20.985643 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:54:20.985643 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:54:20.985643 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:54:20.985643 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:54:20.985643 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 26 14:54:20.985643 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:54:20.985643 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 26 14:54:20.985643 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:54:20.985643 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:54:20.986409 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:54:20.986409 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 26 14:54:20.986409 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:54:20.986409 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 26 14:54:20.986409 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 14:54:20.986409 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 14:54:20.986409 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 14:54:20.986409 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 14:54:20.986409 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 14:54:20.986409 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 14:54:20.986409 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 26 14:54:20.986409 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 26 14:54:20.986409 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 80/166] 10.222.0.46 () {56 vars in 1099 bytes} [Thu May 26 14:54:20 2022] GET /identity/v3/roles/member => generated 84 bytes in 383 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:54:20.987309 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16152c77-5129-4539-bbae-ce16365ed78d None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:20.987870 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16152c77-5129-4539-bbae-ce16365ed78d None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:20.987870 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16152c77-5129-4539-bbae-ce16365ed78d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:21.019920 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-16152c77-5129-4539-bbae-ce16365ed78d None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:54:21.021131 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-16152c77-5129-4539-bbae-ce16365ed78d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:54:21.027477 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a2c5f9-deb8-4960-86ce-cc789eb51156 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:21.137504 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 81/167] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 14:54:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 14:54:21.138688 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16152c77-5129-4539-bbae-ce16365ed78d None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:21.138983 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43b237a-c511-48de-8ed1-62f8c1d48a92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:21.139666 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43b237a-c511-48de-8ed1-62f8c1d48a92 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:21.139950 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43b237a-c511-48de-8ed1-62f8c1d48a92 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:21.140223 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43b237a-c511-48de-8ed1-62f8c1d48a92 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:21.141138 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43b237a-c511-48de-8ed1-62f8c1d48a92 None admin] RBAC: Adding query filter params (role.id=a2e551773cbd4df084020676865b8222, scope.project.id=e474420e5c124a3f86383fce1ef4c4fb, user.id=dede0ef3af5b453796bce68732b5fd29) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:54:21.141477 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43b237a-c511-48de-8ed1-62f8c1d48a92 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:54:21.142996 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43b237a-c511-48de-8ed1-62f8c1d48a92 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:54:21.145353 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:21.157675 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 87/168] 10.222.0.46 () {56 vars in 1374 bytes} [Thu May 26 14:54:20 2022] GET /identity/v3/role_assignments?user.id=dede0ef3af5b453796bce68732b5fd29&scope.project.id=e474420e5c124a3f86383fce1ef4c4fb&role.id=a2e551773cbd4df084020676865b8222 => generated 608 bytes in 412 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:54:21.158600 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43b237a-c511-48de-8ed1-62f8c1d48a92 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:21.339519 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:21.389611 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16152c77-5129-4539-bbae-ce16365ed78d None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:21.405993 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43b237a-c511-48de-8ed1-62f8c1d48a92 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:21.451267 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d71f66-57cf-42cc-a801-abb96537cf62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:21.452085 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d71f66-57cf-42cc-a801-abb96537cf62 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:21.452375 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d71f66-57cf-42cc-a801-abb96537cf62 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:21.452654 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d71f66-57cf-42cc-a801-abb96537cf62 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:21.453523 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d71f66-57cf-42cc-a801-abb96537cf62 None admin] RBAC: Adding query filter params (role.id=d4088d1845a24fbd9bd8975ce0b1d3d0, scope.project.id=e474420e5c124a3f86383fce1ef4c4fb, user.id=70e4c37945d34ae18f6ac64fb237b230) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:54:21.453840 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d71f66-57cf-42cc-a801-abb96537cf62 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:54:21.455327 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d71f66-57cf-42cc-a801-abb96537cf62 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:54:21.468133 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 82/169] 10.222.0.46 () {56 vars in 1374 bytes} [Thu May 26 14:54:21 2022] GET /identity/v3/role_assignments?user.id=70e4c37945d34ae18f6ac64fb237b230&scope.project.id=e474420e5c124a3f86383fce1ef4c4fb&role.id=d4088d1845a24fbd9bd8975ce0b1d3d0 => generated 608 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:54:21.469085 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d71f66-57cf-42cc-a801-abb96537cf62 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:21.501108 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99155364-d729-4de6-b5de-e81d2f0f48f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:21.502169 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99155364-d729-4de6-b5de-e81d2f0f48f9 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:21.502743 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99155364-d729-4de6-b5de-e81d2f0f48f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:21.504115 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99155364-d729-4de6-b5de-e81d2f0f48f9 None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:21.505295 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99155364-d729-4de6-b5de-e81d2f0f48f9 None admin] RBAC: Adding query filter params (name=member) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:54:21.507724 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99155364-d729-4de6-b5de-e81d2f0f48f9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:54:21.509437 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99155364-d729-4de6-b5de-e81d2f0f48f9 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:54:21.523912 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 88/170] 10.222.0.46 () {56 vars in 1101 bytes} [Thu May 26 14:54:21 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 366 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:54:21.529860 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99155364-d729-4de6-b5de-e81d2f0f48f9 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:21.650505 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:21.716721 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d71f66-57cf-42cc-a801-abb96537cf62 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:21.726599 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99155364-d729-4de6-b5de-e81d2f0f48f9 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:21.783135 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae07457-4165-4f62-b9e0-8ea5c8d7d837 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:21.784048 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae07457-4165-4f62-b9e0-8ea5c8d7d837 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:21.784336 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae07457-4165-4f62-b9e0-8ea5c8d7d837 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:21.784613 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae07457-4165-4f62-b9e0-8ea5c8d7d837 None admin] PATH_INFO: `/v3/users/70e4c37945d34ae18f6ac64fb237b230` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:21.786397 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae07457-4165-4f62-b9e0-8ea5c8d7d837 None admin] RBAC: Authorizing `identity:get_user(user_id=70e4c37945d34ae18f6ac64fb237b230)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:54:21.788100 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae07457-4165-4f62-b9e0-8ea5c8d7d837 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:54:21.790188 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 89/171] 10.222.0.46 () {56 vars in 1177 bytes} [Thu May 26 14:54:21 2022] GET /identity/v3/users/70e4c37945d34ae18f6ac64fb237b230 => generated 275 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 14:54:21.795906 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae07457-4165-4f62-b9e0-8ea5c8d7d837 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:21.825707 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c642e2-0226-43dd-8761-a7f0bfb2b35e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:21.827078 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c642e2-0226-43dd-8761-a7f0bfb2b35e None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:21.827521 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c642e2-0226-43dd-8761-a7f0bfb2b35e None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:21.828085 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c642e2-0226-43dd-8761-a7f0bfb2b35e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:21.860989 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c642e2-0226-43dd-8761-a7f0bfb2b35e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 14:54:21.878183 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-10c642e2-0226-43dd-8761-a7f0bfb2b35e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 14:54:21.878183 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:54:21.878183 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:54:21.878183 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:54:21.878183 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:54:21.878183 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:54:21.878183 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:54:21.878183 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:54:21.878183 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:54:21.878183 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:54:21.878183 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:54:21.878183 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:54:21.878183 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 14:54:21.878183 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 14:54:21.878183 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 14:54:21.878183 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 14:54:21.879161 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 14:54:21.879161 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 14:54:21.879161 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 14:54:21.879161 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 14:54:21.879161 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:54:21.879161 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:54:21.879161 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:54:21.879161 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:54:21.879161 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:54:21.879161 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:54:21.879161 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:54:21.879161 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 26 14:54:21.879161 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:54:21.879161 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 26 14:54:21.879161 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:54:21.879161 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:54:21.879958 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:54:21.879958 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 26 14:54:21.879958 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:54:21.879958 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 26 14:54:21.879958 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 14:54:21.879958 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 14:54:21.879958 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 14:54:21.879958 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 14:54:21.879958 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 14:54:21.879958 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 14:54:21.879958 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 14:54:21.879958 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 26 14:54:21.880907 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 83/172] 10.222.0.46 () {56 vars in 1114 bytes} [Thu May 26 14:54:21 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 412 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:54:21.887300 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c642e2-0226-43dd-8761-a7f0bfb2b35e None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:21.897175 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:21.979908 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae07457-4165-4f62-b9e0-8ea5c8d7d837 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:22.043667 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:22.067305 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82433e4-8e1d-436d-9b6a-74f0f5f1e7d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:22.068242 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82433e4-8e1d-436d-9b6a-74f0f5f1e7d4 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:22.068681 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82433e4-8e1d-436d-9b6a-74f0f5f1e7d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:22.069122 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82433e4-8e1d-436d-9b6a-74f0f5f1e7d4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:22.105997 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c642e2-0226-43dd-8761-a7f0bfb2b35e None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:22.176215 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147802c0-2302-4887-9e30-0b85b77b0c73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:22.177249 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147802c0-2302-4887-9e30-0b85b77b0c73 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:22.177745 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147802c0-2302-4887-9e30-0b85b77b0c73 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:22.181932 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147802c0-2302-4887-9e30-0b85b77b0c73 None admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:22.183422 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147802c0-2302-4887-9e30-0b85b77b0c73 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:54:22.184297 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147802c0-2302-4887-9e30-0b85b77b0c73 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:54:22.185728 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147802c0-2302-4887-9e30-0b85b77b0c73 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:54:22.202830 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 84/173] 10.222.0.46 () {56 vars in 1111 bytes} [Thu May 26 14:54:21 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:54:22.208508 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70b5792-e2da-4f9f-bd59-91b92a88ed81 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:22.208960 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70b5792-e2da-4f9f-bd59-91b92a88ed81 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:22.209252 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70b5792-e2da-4f9f-bd59-91b92a88ed81 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:22.210942 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 85/174] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 14:54:22 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 14:54:22.215498 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70b5792-e2da-4f9f-bd59-91b92a88ed81 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:22.223125 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82433e4-8e1d-436d-9b6a-74f0f5f1e7d4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:54:22.225009 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82433e4-8e1d-436d-9b6a-74f0f5f1e7d4 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:54:22.322639 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:22.366490 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70b5792-e2da-4f9f-bd59-91b92a88ed81 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:22.446060 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 90/175] 10.222.0.46 () {58 vars in 1282 bytes} [Thu May 26 14:54:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 645 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 26 14:54:22.448097 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9674a1f2-5e77-4118-8f4e-a856c765eb65 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:22.448393 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9674a1f2-5e77-4118-8f4e-a856c765eb65 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:22.448726 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9674a1f2-5e77-4118-8f4e-a856c765eb65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:22.453415 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6916c123-2c5e-4a5e-9824-bd454d640801 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:22.454439 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6916c123-2c5e-4a5e-9824-bd454d640801 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:22.454912 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6916c123-2c5e-4a5e-9824-bd454d640801 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:22.455349 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6916c123-2c5e-4a5e-9824-bd454d640801 None admin] PATH_INFO: `/v3/users/70e4c37945d34ae18f6ac64fb237b230` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:22.459123 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6916c123-2c5e-4a5e-9824-bd454d640801 None admin] RBAC: Authorizing `identity:get_user(user_id=70e4c37945d34ae18f6ac64fb237b230)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:54:22.464113 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6916c123-2c5e-4a5e-9824-bd454d640801 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:54:22.474065 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 86/176] 10.222.0.46 () {56 vars in 1177 bytes} [Thu May 26 14:54:22 2022] GET /identity/v3/users/70e4c37945d34ae18f6ac64fb237b230 => generated 275 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:54:22.475033 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6916c123-2c5e-4a5e-9824-bd454d640801 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:22.489374 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-9674a1f2-5e77-4118-8f4e-a856c765eb65 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:54:22.491273 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-9674a1f2-5e77-4118-8f4e-a856c765eb65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:54:22.518028 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:22.573079 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 91/177] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 14:54:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 14:54:22.578350 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9674a1f2-5e77-4118-8f4e-a856c765eb65 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:22.607600 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6916c123-2c5e-4a5e-9824-bd454d640801 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:22.645250 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:22.651295 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cd0d79-c2e2-4641-9d1e-65aeadbb44ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:22.652716 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cd0d79-c2e2-4641-9d1e-65aeadbb44ee None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:22.653109 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cd0d79-c2e2-4641-9d1e-65aeadbb44ee None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:22.653508 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cd0d79-c2e2-4641-9d1e-65aeadbb44ee None admin] PATH_INFO: `/v3/projects/1c95d1c3fe20488caf82b4b1125bfbc1` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:22.655553 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51cd0d79-c2e2-4641-9d1e-65aeadbb44ee None admin] RBAC: Authorizing `identity:get_project(project_id=1c95d1c3fe20488caf82b4b1125bfbc1)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:54:22.657715 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51cd0d79-c2e2-4641-9d1e-65aeadbb44ee None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:54:22.660599 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 87/178] 10.222.0.46 () {56 vars in 1186 bytes} [Thu May 26 14:54:22 2022] GET /identity/v3/projects/1c95d1c3fe20488caf82b4b1125bfbc1 => generated 312 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:54:22.662548 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3435a55-1a59-4845-9674-1156d6d43fbe None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:22.662978 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3435a55-1a59-4845-9674-1156d6d43fbe None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:22.663424 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3435a55-1a59-4845-9674-1156d6d43fbe None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:22.665434 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 88/179] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 14:54:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 14:54:22.671123 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ab1c8b-57d7-45b7-a599-bacaf6998dba None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:22.671417 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ab1c8b-57d7-45b7-a599-bacaf6998dba None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:22.671699 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ab1c8b-57d7-45b7-a599-bacaf6998dba None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:22.677248 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 89/180] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 14:54:22 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 14:54:22.677248 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ab1c8b-57d7-45b7-a599-bacaf6998dba None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:22.724913 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9674a1f2-5e77-4118-8f4e-a856c765eb65 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:22.785164 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b384f2ce-6d44-4ce8-abac-904d7ffbb388 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:22.785869 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b384f2ce-6d44-4ce8-abac-904d7ffbb388 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:22.786676 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b384f2ce-6d44-4ce8-abac-904d7ffbb388 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:22.786839 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b384f2ce-6d44-4ce8-abac-904d7ffbb388 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:22.813366 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ab1c8b-57d7-45b7-a599-bacaf6998dba None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:22.824831 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:22.864135 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a76077-f9b3-459f-980a-657582619c10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:22.865423 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a76077-f9b3-459f-980a-657582619c10 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:22.865924 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a76077-f9b3-459f-980a-657582619c10 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:22.866340 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a76077-f9b3-459f-980a-657582619c10 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:22.880908 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a76077-f9b3-459f-980a-657582619c10 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 14:54:22.889375 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-82a76077-f9b3-459f-980a-657582619c10 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 14:54:22.889375 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:54:22.889375 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:54:22.889375 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:54:22.889375 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:54:22.889375 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:54:22.889375 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:54:22.889375 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:54:22.889375 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:54:22.889375 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:54:22.889375 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:54:22.889375 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:54:22.889375 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 14:54:22.889375 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 14:54:22.889375 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 14:54:22.889375 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 14:54:22.890410 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 14:54:22.890410 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 14:54:22.890410 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 14:54:22.890410 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 14:54:22.890410 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:54:22.890410 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:54:22.890410 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:54:22.890410 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:54:22.890410 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:54:22.890410 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:54:22.890410 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:54:22.890410 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 26 14:54:22.890410 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:54:22.890410 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 26 14:54:22.890410 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:54:22.890410 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:54:22.891377 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:54:22.891377 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 26 14:54:22.891377 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:54:22.891377 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 26 14:54:22.891377 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 14:54:22.891377 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 14:54:22.891377 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 14:54:22.891377 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 14:54:22.891377 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 14:54:22.891377 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 14:54:22.891377 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 14:54:22.891377 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 26 14:54:22.892186 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b384f2ce-6d44-4ce8-abac-904d7ffbb388 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:54:22.892268 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 90/181] 10.222.0.46 () {56 vars in 1096 bytes} [Thu May 26 14:54:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:54:22.892268 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a76077-f9b3-459f-980a-657582619c10 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:22.894280 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b384f2ce-6d44-4ce8-abac-904d7ffbb388 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:54:22.975546 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:22.999823 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 92/182] 10.222.0.46 () {58 vars in 1282 bytes} [Thu May 26 14:54:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 14:54:23.002904 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f856daf3-077b-4f8a-bf43-114227c6cb17 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:23.002904 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f856daf3-077b-4f8a-bf43-114227c6cb17 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:23.002904 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f856daf3-077b-4f8a-bf43-114227c6cb17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:23.020199 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-f856daf3-077b-4f8a-bf43-114227c6cb17 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:54:23.021419 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-f856daf3-077b-4f8a-bf43-114227c6cb17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:54:23.023598 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a76077-f9b3-459f-980a-657582619c10 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:23.073078 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db2b328-3376-46e1-b83b-338365eb78b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:23.074104 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db2b328-3376-46e1-b83b-338365eb78b0 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:23.075224 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db2b328-3376-46e1-b83b-338365eb78b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:23.075592 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db2b328-3376-46e1-b83b-338365eb78b0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:23.076942 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db2b328-3376-46e1-b83b-338365eb78b0 None admin] RBAC: Adding query filter params (role.id=d4088d1845a24fbd9bd8975ce0b1d3d0, scope.project.id=1c95d1c3fe20488caf82b4b1125bfbc1, user.id=70e4c37945d34ae18f6ac64fb237b230) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:54:23.077414 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db2b328-3376-46e1-b83b-338365eb78b0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:54:23.079781 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db2b328-3376-46e1-b83b-338365eb78b0 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:54:23.081688 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:23.089433 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 91/183] 10.222.0.46 () {56 vars in 1374 bytes} [Thu May 26 14:54:22 2022] GET /identity/v3/role_assignments?user.id=70e4c37945d34ae18f6ac64fb237b230&scope.project.id=1c95d1c3fe20488caf82b4b1125bfbc1&role.id=d4088d1845a24fbd9bd8975ce0b1d3d0 => generated 608 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:54:23.091437 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14a8c21-71ab-4927-a235-c417cfe28fd6 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:23.091889 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14a8c21-71ab-4927-a235-c417cfe28fd6 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:23.092323 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14a8c21-71ab-4927-a235-c417cfe28fd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:23.106090 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 93/184] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 14:54:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 14:54:23.107665 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f614049-49dc-4096-b028-30ff128ada7d None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:23.108047 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f614049-49dc-4096-b028-30ff128ada7d None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:23.108385 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f614049-49dc-4096-b028-30ff128ada7d None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:23.110195 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 94/185] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 14:54:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 14:54:23.114534 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f614049-49dc-4096-b028-30ff128ada7d None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:23.126890 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-b14a8c21-71ab-4927-a235-c417cfe28fd6 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:54:23.128122 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-b14a8c21-71ab-4927-a235-c417cfe28fd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:54:23.213426 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 92/186] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 14:54:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 14:54:23.214344 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14a8c21-71ab-4927-a235-c417cfe28fd6 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:23.262866 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f614049-49dc-4096-b028-30ff128ada7d None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:23.290828 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:23.312849 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39839800-c6a6-453f-8acd-55454ae69ad9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:23.315076 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39839800-c6a6-453f-8acd-55454ae69ad9 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:23.315076 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39839800-c6a6-453f-8acd-55454ae69ad9 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:23.315076 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39839800-c6a6-453f-8acd-55454ae69ad9 None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:23.315299 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39839800-c6a6-453f-8acd-55454ae69ad9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:54:23.315811 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39839800-c6a6-453f-8acd-55454ae69ad9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:54:23.318149 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39839800-c6a6-453f-8acd-55454ae69ad9 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:54:23.330640 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 95/187] 10.222.0.46 () {56 vars in 1099 bytes} [Thu May 26 14:54:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:54:23.331645 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed3748a-3bcf-4bde-bfed-224b87c82c1a None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:23.331941 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed3748a-3bcf-4bde-bfed-224b87c82c1a None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:23.332222 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed3748a-3bcf-4bde-bfed-224b87c82c1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:23.349293 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-bed3748a-3bcf-4bde-bfed-224b87c82c1a None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:54:23.350553 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-bed3748a-3bcf-4bde-bfed-224b87c82c1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:54:23.376764 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b14a8c21-71ab-4927-a235-c417cfe28fd6 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:23.421014 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:23.427121 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d15cf5-af46-45bd-a4b7-9b9c742c77bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:23.428028 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d15cf5-af46-45bd-a4b7-9b9c742c77bd None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:23.428295 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d15cf5-af46-45bd-a4b7-9b9c742c77bd None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:23.428574 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d15cf5-af46-45bd-a4b7-9b9c742c77bd None admin] PATH_INFO: `/v3/projects/e474420e5c124a3f86383fce1ef4c4fb` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:23.429863 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d15cf5-af46-45bd-a4b7-9b9c742c77bd None admin] RBAC: Authorizing `identity:get_project(project_id=e474420e5c124a3f86383fce1ef4c4fb)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:54:23.431326 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d15cf5-af46-45bd-a4b7-9b9c742c77bd None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:54:23.440523 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 93/188] 10.222.0.46 () {56 vars in 1186 bytes} [Thu May 26 14:54:23 2022] GET /identity/v3/projects/e474420e5c124a3f86383fce1ef4c4fb => generated 298 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:54:23.440523 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d15cf5-af46-45bd-a4b7-9b9c742c77bd None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:23.447812 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 96/189] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 14:54:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 14:54:23.448669 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed3748a-3bcf-4bde-bfed-224b87c82c1a None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:23.546396 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d15cf5-af46-45bd-a4b7-9b9c742c77bd None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:23.557768 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed3748a-3bcf-4bde-bfed-224b87c82c1a None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:23.568916 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:23.581471 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab19de70-5eba-44f2-9b86-d9011f5773a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:23.582217 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab19de70-5eba-44f2-9b86-d9011f5773a4 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:23.582495 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab19de70-5eba-44f2-9b86-d9011f5773a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:23.582776 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab19de70-5eba-44f2-9b86-d9011f5773a4 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:23.593295 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab19de70-5eba-44f2-9b86-d9011f5773a4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 14:54:23.593426 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ac33dc-d58a-4f9e-a0da-c4e08920b9fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:23.594166 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ac33dc-d58a-4f9e-a0da-c4e08920b9fe None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:23.594450 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ac33dc-d58a-4f9e-a0da-c4e08920b9fe None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:23.594726 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ac33dc-d58a-4f9e-a0da-c4e08920b9fe None admin] PATH_INFO: `/v3/domains/default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:23.595883 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ac33dc-d58a-4f9e-a0da-c4e08920b9fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:54:23.597340 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ac33dc-d58a-4f9e-a0da-c4e08920b9fe None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:54:23.606129 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 97/190] 10.222.0.46 () {56 vars in 1108 bytes} [Thu May 26 14:54:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:54:23.606129 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ac33dc-d58a-4f9e-a0da-c4e08920b9fe None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:23.606342 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-ab19de70-5eba-44f2-9b86-d9011f5773a4 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 14:54:23.606342 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:54:23.606342 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:54:23.606342 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:54:23.606342 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:54:23.606342 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:54:23.606342 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:54:23.606342 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:54:23.606342 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:54:23.606342 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:54:23.606342 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:54:23.606342 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:54:23.606342 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 14:54:23.606342 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 14:54:23.606342 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 14:54:23.606342 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 14:54:23.607166 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 14:54:23.607166 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 14:54:23.607166 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 14:54:23.607166 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 14:54:23.607166 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:54:23.607166 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:54:23.607166 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:54:23.607166 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:54:23.607166 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:54:23.607166 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:54:23.607166 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:54:23.607166 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 26 14:54:23.607166 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:54:23.607166 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 26 14:54:23.607166 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:54:23.607166 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:54:23.610018 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:54:23.610018 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 26 14:54:23.610018 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:54:23.610018 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 26 14:54:23.610018 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 14:54:23.610018 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 14:54:23.610018 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 14:54:23.610018 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 14:54:23.610018 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 14:54:23.610018 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 14:54:23.610018 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 14:54:23.610018 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 26 14:54:23.610018 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 94/191] 10.222.0.46 () {56 vars in 1114 bytes} [Thu May 26 14:54:23 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:54:23.610018 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab19de70-5eba-44f2-9b86-d9011f5773a4 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:23.659097 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:23.706890 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab19de70-5eba-44f2-9b86-d9011f5773a4 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:23.711251 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ac33dc-d58a-4f9e-a0da-c4e08920b9fe None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:23.745515 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb93e339-530a-41e0-be23-c02724d1eaec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:23.746236 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb93e339-530a-41e0-be23-c02724d1eaec None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:23.746527 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb93e339-530a-41e0-be23-c02724d1eaec None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:23.746808 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb93e339-530a-41e0-be23-c02724d1eaec None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:23.747640 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb93e339-530a-41e0-be23-c02724d1eaec None admin] RBAC: Adding query filter params (role.id=160115b2143c4b389d39cd93f43e1d01, scope.project.id=e474420e5c124a3f86383fce1ef4c4fb, user.id=70e4c37945d34ae18f6ac64fb237b230) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:54:23.747956 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb93e339-530a-41e0-be23-c02724d1eaec None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:54:23.749201 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5864797f-aedc-4eca-8ae1-bbb16d41c523 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:23.749410 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb93e339-530a-41e0-be23-c02724d1eaec None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:54:23.750122 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5864797f-aedc-4eca-8ae1-bbb16d41c523 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:23.750466 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5864797f-aedc-4eca-8ae1-bbb16d41c523 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:23.751732 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5864797f-aedc-4eca-8ae1-bbb16d41c523 None admin] PATH_INFO: `/v3/users/bde8615a74bc494d9b747dffaa2f3b94` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:23.757462 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 95/192] 10.222.0.46 () {56 vars in 1374 bytes} [Thu May 26 14:54:23 2022] GET /identity/v3/role_assignments?user.id=70e4c37945d34ae18f6ac64fb237b230&scope.project.id=e474420e5c124a3f86383fce1ef4c4fb&role.id=160115b2143c4b389d39cd93f43e1d01 => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:54:23.758317 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb93e339-530a-41e0-be23-c02724d1eaec None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:23.769872 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5864797f-aedc-4eca-8ae1-bbb16d41c523 None admin] RBAC: Authorizing `identity:get_user(user_id=bde8615a74bc494d9b747dffaa2f3b94)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:54:23.771556 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5864797f-aedc-4eca-8ae1-bbb16d41c523 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:54:23.773407 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 98/193] 10.222.0.46 () {56 vars in 1177 bytes} [Thu May 26 14:54:23 2022] GET /identity/v3/users/bde8615a74bc494d9b747dffaa2f3b94 => generated 283 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:54:23.774190 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5864797f-aedc-4eca-8ae1-bbb16d41c523 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:23.821968 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:23.886528 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb93e339-530a-41e0-be23-c02724d1eaec None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:23.900371 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5864797f-aedc-4eca-8ae1-bbb16d41c523 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:23.910816 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:23.933665 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20468f59-9878-4eec-9c55-8eb3018dcf25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:23.934876 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20468f59-9878-4eec-9c55-8eb3018dcf25 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:23.935379 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20468f59-9878-4eec-9c55-8eb3018dcf25 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:23.935796 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20468f59-9878-4eec-9c55-8eb3018dcf25 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:23.945131 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e337eaf-164f-454f-83f4-e3a20f5b1f40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:23.945423 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e337eaf-164f-454f-83f4-e3a20f5b1f40 None admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:23.945709 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e337eaf-164f-454f-83f4-e3a20f5b1f40 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:23.945984 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e337eaf-164f-454f-83f4-e3a20f5b1f40 None admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:23.946893 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e337eaf-164f-454f-83f4-e3a20f5b1f40 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:54:23.949988 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e337eaf-164f-454f-83f4-e3a20f5b1f40 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:54:23.952012 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20468f59-9878-4eec-9c55-8eb3018dcf25 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 14:54:23.960285 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-20468f59-9878-4eec-9c55-8eb3018dcf25 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 26 14:54:23.960285 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:54:23.960285 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:54:23.960285 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:54:23.960285 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:54:23.960285 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:54:23.960285 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:54:23.960285 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:54:23.960285 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:54:23.960285 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:54:23.960285 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:54:23.960285 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:54:23.960285 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 14:54:23.960285 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 14:54:23.960285 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 14:54:23.960285 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 14:54:23.961204 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 14:54:23.961204 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 14:54:23.961204 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 14:54:23.961204 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 14:54:23.961204 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:54:23.961204 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:54:23.961204 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:54:23.961204 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:54:23.961204 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:54:23.961204 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:54:23.961204 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:54:23.961204 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 26 14:54:23.961204 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:54:23.961204 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 26 14:54:23.961204 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:54:23.961204 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:54:23.962035 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:54:23.962035 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 26 14:54:23.962035 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:54:23.962035 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 26 14:54:23.962035 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 14:54:23.962035 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 14:54:23.962035 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 14:54:23.962035 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 14:54:23.962035 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 14:54:23.962035 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 14:54:23.962035 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 26 14:54:23.962035 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 26 14:54:23.962035 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 96/194] 10.222.0.46 () {56 vars in 1099 bytes} [Thu May 26 14:54:23 2022] GET /identity/v3/roles/reader => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:54:23.963476 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20468f59-9878-4eec-9c55-8eb3018dcf25 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:23.981012 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 99/195] 10.222.0.46 () {60 vars in 1132 bytes} [Thu May 26 14:54:23 2022] POST /identity/v3/users => generated 297 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 14:54:23.989122 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e337eaf-164f-454f-83f4-e3a20f5b1f40 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:24.106427 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20468f59-9878-4eec-9c55-8eb3018dcf25 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:24.115457 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:24.132387 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e337eaf-164f-454f-83f4-e3a20f5b1f40 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:24.146987 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda4db51-a67f-4142-87f3-6b7a295803c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:24.147291 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda4db51-a67f-4142-87f3-6b7a295803c0 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:24.147568 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda4db51-a67f-4142-87f3-6b7a295803c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:24.147829 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda4db51-a67f-4142-87f3-6b7a295803c0 None admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:24.148589 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda4db51-a67f-4142-87f3-6b7a295803c0 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:54:24.148898 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda4db51-a67f-4142-87f3-6b7a295803c0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:54:24.150388 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda4db51-a67f-4142-87f3-6b7a295803c0 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:54:24.160008 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 97/196] 10.222.0.46 () {56 vars in 1111 bytes} [Thu May 26 14:54:23 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:54:24.160702 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda4db51-a67f-4142-87f3-6b7a295803c0 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:24.180847 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cda4803-cde1-4348-8c1f-4960b16ea0ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:24.181558 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cda4803-cde1-4348-8c1f-4960b16ea0ad None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:24.181848 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cda4803-cde1-4348-8c1f-4960b16ea0ad None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:24.182128 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cda4803-cde1-4348-8c1f-4960b16ea0ad None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:24.219234 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:24.271558 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cda4803-cde1-4348-8c1f-4960b16ea0ad None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:54:24.273181 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cda4803-cde1-4348-8c1f-4960b16ea0ad None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:54:24.287625 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda4db51-a67f-4142-87f3-6b7a295803c0 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:24.336497 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338290db-acee-4bae-8375-a227f8b8d53e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:24.337910 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338290db-acee-4bae-8375-a227f8b8d53e None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:24.338401 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338290db-acee-4bae-8375-a227f8b8d53e None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:24.338878 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338290db-acee-4bae-8375-a227f8b8d53e None admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:24.340394 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338290db-acee-4bae-8375-a227f8b8d53e None admin] RBAC: Adding query filter params (name=reader) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:54:24.340922 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338290db-acee-4bae-8375-a227f8b8d53e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:54:24.343029 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338290db-acee-4bae-8375-a227f8b8d53e None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:54:24.354006 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 98/197] 10.222.0.46 () {56 vars in 1101 bytes} [Thu May 26 14:54:24 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:54:24.355899 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338290db-acee-4bae-8375-a227f8b8d53e None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:24.398061 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:24.407484 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 100/198] 10.222.0.46 () {58 vars in 1282 bytes} [Thu May 26 14:54:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 14:54:24.408326 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cda4803-cde1-4348-8c1f-4960b16ea0ad None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:24.476140 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338290db-acee-4bae-8375-a227f8b8d53e None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:24.483975 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:24.519987 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6770f0f8-ec0b-47d5-a871-1acf8035782b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:24.520887 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6770f0f8-ec0b-47d5-a871-1acf8035782b None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:24.521158 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6770f0f8-ec0b-47d5-a871-1acf8035782b None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:24.521429 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6770f0f8-ec0b-47d5-a871-1acf8035782b None admin] PATH_INFO: `/v3/users/bde8615a74bc494d9b747dffaa2f3b94` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:24.522745 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6770f0f8-ec0b-47d5-a871-1acf8035782b None admin] RBAC: Authorizing `identity:get_user(user_id=bde8615a74bc494d9b747dffaa2f3b94)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:54:24.524319 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6770f0f8-ec0b-47d5-a871-1acf8035782b None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:54:24.525982 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 99/199] 10.222.0.46 () {56 vars in 1177 bytes} [Thu May 26 14:54:24 2022] GET /identity/v3/users/bde8615a74bc494d9b747dffaa2f3b94 => generated 283 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:54:24.531222 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6770f0f8-ec0b-47d5-a871-1acf8035782b None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:24.536749 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cda4803-cde1-4348-8c1f-4960b16ea0ad None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:24.586869 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693e10af-7559-4450-bd01-1ba736a78628 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:24.587916 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693e10af-7559-4450-bd01-1ba736a78628 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:24.588201 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693e10af-7559-4450-bd01-1ba736a78628 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:24.588478 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693e10af-7559-4450-bd01-1ba736a78628 None admin] PATH_INFO: `/v3/users/990e3996e18d4a1aa532413cc2dbc3a1` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:24.590296 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693e10af-7559-4450-bd01-1ba736a78628 None admin] RBAC: Authorizing `identity:get_user(user_id=990e3996e18d4a1aa532413cc2dbc3a1)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:54:24.592048 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693e10af-7559-4450-bd01-1ba736a78628 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:54:24.594517 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 101/200] 10.222.0.46 () {56 vars in 1177 bytes} [Thu May 26 14:54:24 2022] GET /identity/v3/users/990e3996e18d4a1aa532413cc2dbc3a1 => generated 289 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:54:24.595341 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693e10af-7559-4450-bd01-1ba736a78628 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:24.663881 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:24.669792 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6770f0f8-ec0b-47d5-a871-1acf8035782b None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:24.717783 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6b41ac-b500-4d0b-ad5a-8e2f1cb21a7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:24.719692 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6b41ac-b500-4d0b-ad5a-8e2f1cb21a7c None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:24.720210 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6b41ac-b500-4d0b-ad5a-8e2f1cb21a7c None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:24.720934 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6b41ac-b500-4d0b-ad5a-8e2f1cb21a7c None admin] PATH_INFO: `/v3/projects/94ad8a7442da4d658f58d1543ae06493` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:24.722986 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6b41ac-b500-4d0b-ad5a-8e2f1cb21a7c None admin] RBAC: Authorizing `identity:get_project(project_id=94ad8a7442da4d658f58d1543ae06493)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:54:24.724253 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693e10af-7559-4450-bd01-1ba736a78628 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:24.724922 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6b41ac-b500-4d0b-ad5a-8e2f1cb21a7c None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:54:24.727994 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 100/201] 10.222.0.46 () {56 vars in 1186 bytes} [Thu May 26 14:54:24 2022] GET /identity/v3/projects/94ad8a7442da4d658f58d1543ae06493 => generated 302 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:54:24.728779 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6b41ac-b500-4d0b-ad5a-8e2f1cb21a7c None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:24.763182 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:24.770366 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7ff449-1b46-49c8-b316-1e746c48305c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:24.771250 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7ff449-1b46-49c8-b316-1e746c48305c None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:24.771832 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7ff449-1b46-49c8-b316-1e746c48305c None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:24.772213 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7ff449-1b46-49c8-b316-1e746c48305c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:24.847205 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6b41ac-b500-4d0b-ad5a-8e2f1cb21a7c None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:24.856308 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7ff449-1b46-49c8-b316-1e746c48305c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:54:24.858470 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7ff449-1b46-49c8-b316-1e746c48305c None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:54:24.884040 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7961304d-63a0-4597-8296-e6673597f039 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:24.884797 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7961304d-63a0-4597-8296-e6673597f039 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:24.885083 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7961304d-63a0-4597-8296-e6673597f039 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:24.885358 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7961304d-63a0-4597-8296-e6673597f039 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:24.924002 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:24.963726 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7961304d-63a0-4597-8296-e6673597f039 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:54:24.966192 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7961304d-63a0-4597-8296-e6673597f039 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:54:24.971765 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 102/202] 10.222.0.46 () {58 vars in 1282 bytes} [Thu May 26 14:54:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 14:54:24.976462 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7ff449-1b46-49c8-b316-1e746c48305c None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:25.004187 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:25.082246 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7ff449-1b46-49c8-b316-1e746c48305c None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:25.088558 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 101/203] 10.222.0.46 () {58 vars in 1282 bytes} [Thu May 26 14:54:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 14:54:25.089762 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7961304d-63a0-4597-8296-e6673597f039 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:25.118856 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bea2fb-95bc-4aa5-b458-0827af2e5e89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:25.119668 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bea2fb-95bc-4aa5-b458-0827af2e5e89 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:25.120061 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bea2fb-95bc-4aa5-b458-0827af2e5e89 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:25.120446 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bea2fb-95bc-4aa5-b458-0827af2e5e89 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:25.122002 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3bea2fb-95bc-4aa5-b458-0827af2e5e89 None admin] RBAC: Adding query filter params (role.id=a2e551773cbd4df084020676865b8222, scope.project.id=94ad8a7442da4d658f58d1543ae06493, user.id=bde8615a74bc494d9b747dffaa2f3b94) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:54:25.122421 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3bea2fb-95bc-4aa5-b458-0827af2e5e89 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:54:25.123978 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3bea2fb-95bc-4aa5-b458-0827af2e5e89 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:54:25.130141 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 103/204] 10.222.0.46 () {56 vars in 1374 bytes} [Thu May 26 14:54:24 2022] GET /identity/v3/role_assignments?user.id=bde8615a74bc494d9b747dffaa2f3b94&scope.project.id=94ad8a7442da4d658f58d1543ae06493&role.id=a2e551773cbd4df084020676865b8222 => generated 259 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:54:25.138431 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bea2fb-95bc-4aa5-b458-0827af2e5e89 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:25.164811 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:25.239303 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7961304d-63a0-4597-8296-e6673597f039 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:25.282296 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d6b905-e4f1-488f-aa4c-152e3ace2e50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:25.283551 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d6b905-e4f1-488f-aa4c-152e3ace2e50 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:25.283980 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d6b905-e4f1-488f-aa4c-152e3ace2e50 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:25.284411 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d6b905-e4f1-488f-aa4c-152e3ace2e50 None admin] PATH_INFO: `/v3/projects/94ad8a7442da4d658f58d1543ae06493` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:25.285847 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d6b905-e4f1-488f-aa4c-152e3ace2e50 None admin] RBAC: Authorizing `identity:get_project(project_id=94ad8a7442da4d658f58d1543ae06493)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:54:25.287498 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d6b905-e4f1-488f-aa4c-152e3ace2e50 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:54:25.293863 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 102/205] 10.222.0.46 () {56 vars in 1186 bytes} [Thu May 26 14:54:25 2022] GET /identity/v3/projects/94ad8a7442da4d658f58d1543ae06493 => generated 302 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:54:25.296438 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d6b905-e4f1-488f-aa4c-152e3ace2e50 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:25.388354 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:25.399944 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bea2fb-95bc-4aa5-b458-0827af2e5e89 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:25.412296 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d6b905-e4f1-488f-aa4c-152e3ace2e50 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:25.448702 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b912211b-e6ef-4c7a-833f-a01ab129bee8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:25.449653 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b912211b-e6ef-4c7a-833f-a01ab129bee8 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:25.450080 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b912211b-e6ef-4c7a-833f-a01ab129bee8 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:25.450518 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b912211b-e6ef-4c7a-833f-a01ab129bee8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:25.451484 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b912211b-e6ef-4c7a-833f-a01ab129bee8 None admin] RBAC: Adding query filter params (role.id=160115b2143c4b389d39cd93f43e1d01, scope.project.id=94ad8a7442da4d658f58d1543ae06493, user.id=bde8615a74bc494d9b747dffaa2f3b94) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:54:25.451949 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b912211b-e6ef-4c7a-833f-a01ab129bee8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:54:25.453642 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b912211b-e6ef-4c7a-833f-a01ab129bee8 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:54:25.456265 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:25.460617 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d55df6-41a4-4470-89e9-280c412cc113 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:25.462702 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d55df6-41a4-4470-89e9-280c412cc113 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:25.463312 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d55df6-41a4-4470-89e9-280c412cc113 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:25.463912 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d55df6-41a4-4470-89e9-280c412cc113 None admin] PATH_INFO: `/v3/projects/e474420e5c124a3f86383fce1ef4c4fb` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:25.465363 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 103/206] 10.222.0.46 () {56 vars in 1374 bytes} [Thu May 26 14:54:25 2022] GET /identity/v3/role_assignments?user.id=bde8615a74bc494d9b747dffaa2f3b94&scope.project.id=94ad8a7442da4d658f58d1543ae06493&role.id=160115b2143c4b389d39cd93f43e1d01 => generated 259 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:54:25.466450 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d55df6-41a4-4470-89e9-280c412cc113 None admin] RBAC: Authorizing `identity:get_project(project_id=e474420e5c124a3f86383fce1ef4c4fb)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:54:25.469063 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d55df6-41a4-4470-89e9-280c412cc113 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:54:25.472653 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 104/207] 10.222.0.46 () {56 vars in 1186 bytes} [Thu May 26 14:54:25 2022] GET /identity/v3/projects/e474420e5c124a3f86383fce1ef4c4fb => generated 298 bytes in 342 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:54:25.478163 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b912211b-e6ef-4c7a-833f-a01ab129bee8 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:25.603653 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b912211b-e6ef-4c7a-833f-a01ab129bee8 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:25.651232 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed4a475-2f7a-49df-8aee-26769780b87f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:25.651912 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed4a475-2f7a-49df-8aee-26769780b87f None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:25.652225 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed4a475-2f7a-49df-8aee-26769780b87f None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:25.652553 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed4a475-2f7a-49df-8aee-26769780b87f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:25.653405 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed4a475-2f7a-49df-8aee-26769780b87f None admin] RBAC: Adding query filter params (role.id=adb2011fe8f74ecbbb94465253e76c1c, scope.project.id=e474420e5c124a3f86383fce1ef4c4fb, user.id=990e3996e18d4a1aa532413cc2dbc3a1) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:54:25.653744 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed4a475-2f7a-49df-8aee-26769780b87f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:54:25.655298 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed4a475-2f7a-49df-8aee-26769780b87f None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:54:25.662404 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 104/208] 10.222.0.46 () {56 vars in 1374 bytes} [Thu May 26 14:54:25 2022] GET /identity/v3/role_assignments?user.id=990e3996e18d4a1aa532413cc2dbc3a1&scope.project.id=e474420e5c124a3f86383fce1ef4c4fb&role.id=adb2011fe8f74ecbbb94465253e76c1c => generated 608 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:54:26.530894 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb0a72c-e8dc-4db6-82d7-90aed7619a9e None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:26.531897 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb0a72c-e8dc-4db6-82d7-90aed7619a9e None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:26.532392 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb0a72c-e8dc-4db6-82d7-90aed7619a9e None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:26.534285 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 105/209] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 14:54:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 14:54:26.540326 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7048fa6a-c2c7-4b39-a324-53051997db7e None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:26.541266 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7048fa6a-c2c7-4b39-a324-53051997db7e None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:26.541817 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7048fa6a-c2c7-4b39-a324-53051997db7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:26.563713 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-7048fa6a-c2c7-4b39-a324-53051997db7e None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:54:26.565293 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-7048fa6a-c2c7-4b39-a324-53051997db7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:54:26.653233 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:26.669897 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 105/210] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 14:54:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 14:54:26.692656 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb0a72c-e8dc-4db6-82d7-90aed7619a9e None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:26.755047 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66060d05-2a8e-4cf1-9b31-3e98fb409b2e None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:26.755800 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66060d05-2a8e-4cf1-9b31-3e98fb409b2e None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:26.756145 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66060d05-2a8e-4cf1-9b31-3e98fb409b2e None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:26.757883 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 106/211] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 14:54:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 14:54:26.763390 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338af74a-107c-41a0-a32c-1bd014df4f06 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:26.763539 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338af74a-107c-41a0-a32c-1bd014df4f06 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:26.763786 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338af74a-107c-41a0-a32c-1bd014df4f06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:26.781537 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-338af74a-107c-41a0-a32c-1bd014df4f06 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:54:26.782639 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-338af74a-107c-41a0-a32c-1bd014df4f06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:54:26.809729 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb0a72c-e8dc-4db6-82d7-90aed7619a9e None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:26.846899 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73dc07b1-d2ef-4f10-9177-35f2c2ec1355 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:26.848082 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dc07b1-d2ef-4f10-9177-35f2c2ec1355 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:26.848570 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dc07b1-d2ef-4f10-9177-35f2c2ec1355 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:26.849279 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dc07b1-d2ef-4f10-9177-35f2c2ec1355 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:26.857874 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 107/212] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 14:54:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 14:54:26.862424 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73dc07b1-d2ef-4f10-9177-35f2c2ec1355 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 14:54:26.870935 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-73dc07b1-d2ef-4f10-9177-35f2c2ec1355 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 26 14:54:26.870935 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:54:26.870935 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:54:26.870935 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:54:26.870935 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:54:26.870935 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:54:26.870935 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:54:26.870935 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:54:26.870935 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:54:26.870935 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:54:26.870935 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:54:26.870935 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:54:26.870935 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 14:54:26.870935 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 14:54:26.870935 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 14:54:26.870935 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 14:54:26.871816 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 14:54:26.871816 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 14:54:26.871816 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 14:54:26.871816 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 14:54:26.871816 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:54:26.871816 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:54:26.871816 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:54:26.871816 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:54:26.871816 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:54:26.871816 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:54:26.871816 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:54:26.871816 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 26 14:54:26.871816 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:54:26.871816 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 26 14:54:26.871816 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:54:26.871816 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:54:26.872594 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:54:26.872594 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 26 14:54:26.872594 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:54:26.872594 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 26 14:54:26.872594 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 14:54:26.872594 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 14:54:26.872594 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 14:54:26.872594 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 14:54:26.872594 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 14:54:26.872594 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 14:54:26.872594 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 26 14:54:26.872594 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 26 14:54:26.875244 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 106/213] 10.222.0.46 () {56 vars in 1099 bytes} [Thu May 26 14:54:26 2022] GET /identity/v3/roles/member => generated 84 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:54:26.881433 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338af74a-107c-41a0-a32c-1bd014df4f06 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:26.883073 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73dc07b1-d2ef-4f10-9177-35f2c2ec1355 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:26.910360 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:26.991150 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73dc07b1-d2ef-4f10-9177-35f2c2ec1355 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:26.995365 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338af74a-107c-41a0-a32c-1bd014df4f06 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:27.029292 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1cb588-0366-4641-af9d-f8f5d621fafe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:27.030509 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1cb588-0366-4641-af9d-f8f5d621fafe None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:27.030999 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1cb588-0366-4641-af9d-f8f5d621fafe None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:27.031474 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1cb588-0366-4641-af9d-f8f5d621fafe None admin] PATH_INFO: `/v3/domains/default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:27.032332 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:27.033989 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1cb588-0366-4641-af9d-f8f5d621fafe None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:54:27.035708 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1cb588-0366-4641-af9d-f8f5d621fafe None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:54:27.038210 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 107/214] 10.222.0.46 () {56 vars in 1108 bytes} [Thu May 26 14:54:26 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:54:27.045761 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1cb588-0366-4641-af9d-f8f5d621fafe None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:27.055978 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d157e5-61d0-429b-a124-228d803aa9e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:27.056561 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d157e5-61d0-429b-a124-228d803aa9e1 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:27.056991 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d157e5-61d0-429b-a124-228d803aa9e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:27.057431 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d157e5-61d0-429b-a124-228d803aa9e1 None admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:27.058716 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d157e5-61d0-429b-a124-228d803aa9e1 None admin] RBAC: Adding query filter params (name=member) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:54:27.059182 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d157e5-61d0-429b-a124-228d803aa9e1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:54:27.061494 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d157e5-61d0-429b-a124-228d803aa9e1 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:54:27.076338 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 108/215] 10.222.0.46 () {56 vars in 1101 bytes} [Thu May 26 14:54:26 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:54:27.084176 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d157e5-61d0-429b-a124-228d803aa9e1 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:27.153350 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:27.164201 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1cb588-0366-4641-af9d-f8f5d621fafe None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:27.202579 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386be14a-770d-4fd5-a3a9-53bf20d92be0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:27.203736 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386be14a-770d-4fd5-a3a9-53bf20d92be0 None admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:27.204224 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386be14a-770d-4fd5-a3a9-53bf20d92be0 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:27.204702 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386be14a-770d-4fd5-a3a9-53bf20d92be0 None admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:27.205837 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386be14a-770d-4fd5-a3a9-53bf20d92be0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:54:27.207721 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386be14a-770d-4fd5-a3a9-53bf20d92be0 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:54:27.209755 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d157e5-61d0-429b-a124-228d803aa9e1 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:27.233768 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 108/216] 10.222.0.46 () {60 vars in 1132 bytes} [Thu May 26 14:54:27 2022] POST /identity/v3/users => generated 297 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 14:54:27.266043 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fcb3711-6e71-40bc-8267-5ed883bd20f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:27.267505 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcb3711-6e71-40bc-8267-5ed883bd20f5 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:27.267998 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcb3711-6e71-40bc-8267-5ed883bd20f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:27.268509 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fcb3711-6e71-40bc-8267-5ed883bd20f5 None admin] PATH_INFO: `/v3/users/4bddaf6a73f244baa66f51396d56e01f` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:27.286530 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fcb3711-6e71-40bc-8267-5ed883bd20f5 None admin] RBAC: Authorizing `identity:get_user(user_id=4bddaf6a73f244baa66f51396d56e01f)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:54:27.288563 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fcb3711-6e71-40bc-8267-5ed883bd20f5 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:54:27.290965 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 109/217] 10.222.0.46 () {56 vars in 1177 bytes} [Thu May 26 14:54:27 2022] GET /identity/v3/users/4bddaf6a73f244baa66f51396d56e01f => generated 297 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 14:54:27.296552 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386be14a-770d-4fd5-a3a9-53bf20d92be0 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:27.434648 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386be14a-770d-4fd5-a3a9-53bf20d92be0 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:27.473113 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def46281-6cf9-426b-a05a-ed39a5dde16e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:27.474065 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def46281-6cf9-426b-a05a-ed39a5dde16e None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:27.474513 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def46281-6cf9-426b-a05a-ed39a5dde16e None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:27.474891 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def46281-6cf9-426b-a05a-ed39a5dde16e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:27.494022 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:27.546819 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b605811-2721-493b-906a-2a5eeb86c161 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:27.547322 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b605811-2721-493b-906a-2a5eeb86c161 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:27.547464 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b605811-2721-493b-906a-2a5eeb86c161 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:27.549031 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 110/218] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 14:54:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 14:54:27.554847 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52951e31-f8ad-4371-9b60-6f4ac3f9f3fc None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:27.555131 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52951e31-f8ad-4371-9b60-6f4ac3f9f3fc None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:27.555405 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52951e31-f8ad-4371-9b60-6f4ac3f9f3fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:27.563496 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def46281-6cf9-426b-a05a-ed39a5dde16e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:54:27.565014 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def46281-6cf9-426b-a05a-ed39a5dde16e None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:54:27.581312 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-52951e31-f8ad-4371-9b60-6f4ac3f9f3fc None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:54:27.582681 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-52951e31-f8ad-4371-9b60-6f4ac3f9f3fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:54:27.648467 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:27.687415 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 111/219] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 14:54:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 14:54:27.703358 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 109/220] 10.222.0.46 () {58 vars in 1282 bytes} [Thu May 26 14:54:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 14:54:27.708795 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52951e31-f8ad-4371-9b60-6f4ac3f9f3fc None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:27.711033 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def46281-6cf9-426b-a05a-ed39a5dde16e None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:27.786775 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:27.837236 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52951e31-f8ad-4371-9b60-6f4ac3f9f3fc None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:27.900813 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498f8fbf-794a-4a0e-ba09-e459fcb2e5c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:27.901712 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498f8fbf-794a-4a0e-ba09-e459fcb2e5c0 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:27.901992 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498f8fbf-794a-4a0e-ba09-e459fcb2e5c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:27.902824 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498f8fbf-794a-4a0e-ba09-e459fcb2e5c0 None admin] PATH_INFO: `/v3/projects/94ad8a7442da4d658f58d1543ae06493` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:27.903999 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498f8fbf-794a-4a0e-ba09-e459fcb2e5c0 None admin] RBAC: Authorizing `identity:get_project(project_id=94ad8a7442da4d658f58d1543ae06493)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:54:27.905634 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498f8fbf-794a-4a0e-ba09-e459fcb2e5c0 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:54:27.906723 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def46281-6cf9-426b-a05a-ed39a5dde16e None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:27.907586 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 112/221] 10.222.0.46 () {56 vars in 1186 bytes} [Thu May 26 14:54:27 2022] GET /identity/v3/projects/94ad8a7442da4d658f58d1543ae06493 => generated 302 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 14:54:27.915194 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498f8fbf-794a-4a0e-ba09-e459fcb2e5c0 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:27.952450 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a2ecd8-022b-42a7-931a-b99ccc8c4d16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:27.953677 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a2ecd8-022b-42a7-931a-b99ccc8c4d16 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:27.955922 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a2ecd8-022b-42a7-931a-b99ccc8c4d16 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:27.957279 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a2ecd8-022b-42a7-931a-b99ccc8c4d16 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:27.982146 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a2ecd8-022b-42a7-931a-b99ccc8c4d16 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 14:54:27.999678 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:28.002049 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-06a2ecd8-022b-42a7-931a-b99ccc8c4d16 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 14:54:28.002049 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:54:28.002049 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:54:28.002049 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:54:28.002049 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:54:28.002049 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:54:28.002049 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:54:28.002049 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:54:28.002049 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:54:28.002049 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:54:28.002049 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:54:28.002049 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:54:28.002049 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 14:54:28.002049 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 14:54:28.002049 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 14:54:28.002049 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 14:54:28.002928 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 14:54:28.002928 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 14:54:28.002928 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 14:54:28.002928 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 14:54:28.002928 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:54:28.002928 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:54:28.002928 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:54:28.002928 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:54:28.002928 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:54:28.002928 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:54:28.002928 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:54:28.002928 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 26 14:54:28.002928 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:54:28.002928 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 26 14:54:28.002928 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:54:28.002928 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:54:28.003701 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:54:28.003701 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 26 14:54:28.003701 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:54:28.003701 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 26 14:54:28.003701 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 14:54:28.003701 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 14:54:28.003701 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 14:54:28.003701 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 14:54:28.003701 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 14:54:28.003701 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 14:54:28.003701 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 14:54:28.003701 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 26 14:54:28.004312 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 110/222] 10.222.0.46 () {56 vars in 1096 bytes} [Thu May 26 14:54:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 14:54:28.008518 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a2ecd8-022b-42a7-931a-b99ccc8c4d16 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:28.092002 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498f8fbf-794a-4a0e-ba09-e459fcb2e5c0 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:28.166117 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b865298-1ebf-4956-bbd9-2b0b8574aa4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:28.167162 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b865298-1ebf-4956-bbd9-2b0b8574aa4a None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:28.167658 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b865298-1ebf-4956-bbd9-2b0b8574aa4a None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:28.168677 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b865298-1ebf-4956-bbd9-2b0b8574aa4a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:28.170602 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b865298-1ebf-4956-bbd9-2b0b8574aa4a None admin] RBAC: Adding query filter params (role.id=d4088d1845a24fbd9bd8975ce0b1d3d0, scope.project.id=94ad8a7442da4d658f58d1543ae06493, user.id=4bddaf6a73f244baa66f51396d56e01f) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:54:28.171929 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b865298-1ebf-4956-bbd9-2b0b8574aa4a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:54:28.173611 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b865298-1ebf-4956-bbd9-2b0b8574aa4a None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:54:28.187751 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 113/223] 10.222.0.46 () {56 vars in 1374 bytes} [Thu May 26 14:54:27 2022] GET /identity/v3/role_assignments?user.id=4bddaf6a73f244baa66f51396d56e01f&scope.project.id=94ad8a7442da4d658f58d1543ae06493&role.id=d4088d1845a24fbd9bd8975ce0b1d3d0 => generated 259 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 14:54:28.195080 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a2ecd8-022b-42a7-931a-b99ccc8c4d16 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:28.203717 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:28.231898 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0928ff-edd4-42e5-bf45-38f11ccd84fd None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:28.232208 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0928ff-edd4-42e5-bf45-38f11ccd84fd None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:28.232487 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0928ff-edd4-42e5-bf45-38f11ccd84fd None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:28.234162 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 114/224] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 14:54:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 14:54:28.239807 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4770f1a1-ff24-49d0-90a3-f49ad79d5055 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:28.240106 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4770f1a1-ff24-49d0-90a3-f49ad79d5055 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:28.240389 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4770f1a1-ff24-49d0-90a3-f49ad79d5055 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:28.247243 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c797ba-bb8f-4df7-8976-ba92b9f3d8fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:28.248313 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c797ba-bb8f-4df7-8976-ba92b9f3d8fe None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:28.248795 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c797ba-bb8f-4df7-8976-ba92b9f3d8fe None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:28.249266 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c797ba-bb8f-4df7-8976-ba92b9f3d8fe None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:28.250245 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c797ba-bb8f-4df7-8976-ba92b9f3d8fe None admin] RBAC: Adding query filter params (name=admin) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:54:28.250752 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c797ba-bb8f-4df7-8976-ba92b9f3d8fe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:54:28.252360 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c797ba-bb8f-4df7-8976-ba92b9f3d8fe None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:54:28.258802 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-4770f1a1-ff24-49d0-90a3-f49ad79d5055 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:54:28.260008 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-4770f1a1-ff24-49d0-90a3-f49ad79d5055 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:54:28.269083 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 111/225] 10.222.0.46 () {56 vars in 1099 bytes} [Thu May 26 14:54:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 14:54:28.275789 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c797ba-bb8f-4df7-8976-ba92b9f3d8fe None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:28.377568 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 115/226] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 14:54:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 14:54:28.407818 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4770f1a1-ff24-49d0-90a3-f49ad79d5055 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:28.467686 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:28.528978 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c797ba-bb8f-4df7-8976-ba92b9f3d8fe None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:28.570962 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0389e30-8091-4b17-a456-bad6a5d16d62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:28.571843 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0389e30-8091-4b17-a456-bad6a5d16d62 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:28.572155 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0389e30-8091-4b17-a456-bad6a5d16d62 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:28.572376 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0389e30-8091-4b17-a456-bad6a5d16d62 None admin] PATH_INFO: `/v3/users/bde8615a74bc494d9b747dffaa2f3b94` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:28.573618 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0389e30-8091-4b17-a456-bad6a5d16d62 None admin] RBAC: Authorizing `identity:get_user(user_id=bde8615a74bc494d9b747dffaa2f3b94)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:54:28.576319 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0389e30-8091-4b17-a456-bad6a5d16d62 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:54:28.578799 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 112/227] 10.222.0.46 () {56 vars in 1177 bytes} [Thu May 26 14:54:28 2022] GET /identity/v3/users/bde8615a74bc494d9b747dffaa2f3b94 => generated 283 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:54:28.581833 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4770f1a1-ff24-49d0-90a3-f49ad79d5055 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:28.583759 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0389e30-8091-4b17-a456-bad6a5d16d62 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:28.651989 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda4c119-01c3-46a1-abf0-1dc14f27f311 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:28.653228 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:28.654972 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda4c119-01c3-46a1-abf0-1dc14f27f311 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:28.661887 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda4c119-01c3-46a1-abf0-1dc14f27f311 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:28.662181 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda4c119-01c3-46a1-abf0-1dc14f27f311 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:28.691500 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda4c119-01c3-46a1-abf0-1dc14f27f311 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 14:54:28.700414 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-fda4c119-01c3-46a1-abf0-1dc14f27f311 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 14:54:28.700414 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:54:28.700414 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:54:28.700414 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:54:28.700414 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:54:28.700414 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:54:28.700414 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:54:28.700414 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:54:28.700414 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:54:28.700414 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:54:28.700414 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:54:28.700414 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:54:28.700414 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 14:54:28.700414 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 14:54:28.700414 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 14:54:28.700414 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 14:54:28.701305 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 14:54:28.701305 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 14:54:28.701305 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 14:54:28.701305 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 14:54:28.701305 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:54:28.701305 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:54:28.701305 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:54:28.701305 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:54:28.701305 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:54:28.701305 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:54:28.701305 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:54:28.701305 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 26 14:54:28.701305 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:54:28.701305 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 26 14:54:28.701305 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:54:28.701305 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:54:28.702137 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:54:28.702137 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 26 14:54:28.702137 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:54:28.702137 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 26 14:54:28.702137 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 14:54:28.702137 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 14:54:28.702137 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 14:54:28.702137 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 14:54:28.702137 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 14:54:28.702137 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 14:54:28.702137 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 14:54:28.702137 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 26 14:54:28.709790 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 116/228] 10.222.0.46 () {56 vars in 1114 bytes} [Thu May 26 14:54:28 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 14:54:28.716721 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda4c119-01c3-46a1-abf0-1dc14f27f311 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:28.736534 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0389e30-8091-4b17-a456-bad6a5d16d62 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:28.776741 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf5acc6-87fa-46b2-ac00-c70d31b3bd03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:28.778092 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf5acc6-87fa-46b2-ac00-c70d31b3bd03 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:28.778575 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf5acc6-87fa-46b2-ac00-c70d31b3bd03 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:28.779389 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf5acc6-87fa-46b2-ac00-c70d31b3bd03 None admin] PATH_INFO: `/v3/projects/94ad8a7442da4d658f58d1543ae06493` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:28.781081 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf5acc6-87fa-46b2-ac00-c70d31b3bd03 None admin] RBAC: Authorizing `identity:get_project(project_id=94ad8a7442da4d658f58d1543ae06493)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:54:28.783141 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf5acc6-87fa-46b2-ac00-c70d31b3bd03 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:54:28.785692 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 113/229] 10.222.0.46 () {56 vars in 1186 bytes} [Thu May 26 14:54:28 2022] GET /identity/v3/projects/94ad8a7442da4d658f58d1543ae06493 => generated 302 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 14:54:28.791415 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf5acc6-87fa-46b2-ac00-c70d31b3bd03 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:28.929904 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:28.941298 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda4c119-01c3-46a1-abf0-1dc14f27f311 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:28.992056 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2e44a3-9bd2-4f61-97e5-a29ae7d84f24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:28.992410 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf5acc6-87fa-46b2-ac00-c70d31b3bd03 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:28.992736 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2e44a3-9bd2-4f61-97e5-a29ae7d84f24 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:28.993015 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2e44a3-9bd2-4f61-97e5-a29ae7d84f24 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:28.993290 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc2e44a3-9bd2-4f61-97e5-a29ae7d84f24 None admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:28.994215 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2e44a3-9bd2-4f61-97e5-a29ae7d84f24 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:54:28.994493 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2e44a3-9bd2-4f61-97e5-a29ae7d84f24 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:54:28.996410 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc2e44a3-9bd2-4f61-97e5-a29ae7d84f24 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:54:28.999181 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:29.007023 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 117/230] 10.222.0.46 () {56 vars in 1111 bytes} [Thu May 26 14:54:28 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 14:54:29.014752 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2e44a3-9bd2-4f61-97e5-a29ae7d84f24 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:29.090200 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e2a19a-1acc-4880-8cfd-c45e81a24d95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:29.091114 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e2a19a-1acc-4880-8cfd-c45e81a24d95 None admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:29.091403 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e2a19a-1acc-4880-8cfd-c45e81a24d95 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:29.091684 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e2a19a-1acc-4880-8cfd-c45e81a24d95 None admin] PATH_INFO: `/v3/projects/94ad8a7442da4d658f58d1543ae06493/users/bde8615a74bc494d9b747dffaa2f3b94/roles/a2e551773cbd4df084020676865b8222` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:29.098840 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e2a19a-1acc-4880-8cfd-c45e81a24d95 None admin] RBAC: Authorizing `identity:create_grant(project_id=94ad8a7442da4d658f58d1543ae06493, user_id=bde8615a74bc494d9b747dffaa2f3b94, role_id=a2e551773cbd4df084020676865b8222)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:54:29.100305 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e2a19a-1acc-4880-8cfd-c45e81a24d95 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:54:29.119885 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc2e44a3-9bd2-4f61-97e5-a29ae7d84f24 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:29.138087 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 114/231] 10.222.0.46 () {58 vars in 1439 bytes} [Thu May 26 14:54:28 2022] PUT /identity/v3/projects/94ad8a7442da4d658f58d1543ae06493/users/bde8615a74bc494d9b747dffaa2f3b94/roles/a2e551773cbd4df084020676865b8222 => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 14:54:29.160217 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c58f2b-aa4e-49f2-a495-aeedd7987470 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:29.161142 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c58f2b-aa4e-49f2-a495-aeedd7987470 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:29.161426 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c58f2b-aa4e-49f2-a495-aeedd7987470 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:29.161731 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c58f2b-aa4e-49f2-a495-aeedd7987470 None admin] PATH_INFO: `/v3/users/bde8615a74bc494d9b747dffaa2f3b94` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:29.162937 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c58f2b-aa4e-49f2-a495-aeedd7987470 None admin] RBAC: Authorizing `identity:get_user(user_id=bde8615a74bc494d9b747dffaa2f3b94)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:54:29.164493 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c58f2b-aa4e-49f2-a495-aeedd7987470 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:54:29.166153 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 118/232] 10.222.0.46 () {56 vars in 1177 bytes} [Thu May 26 14:54:29 2022] GET /identity/v3/users/bde8615a74bc494d9b747dffaa2f3b94 => generated 283 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 14:54:29.184195 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e2a19a-1acc-4880-8cfd-c45e81a24d95 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:29.339218 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e2a19a-1acc-4880-8cfd-c45e81a24d95 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:29.382585 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48da26e2-1645-4855-87b9-c050f5e22943 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:29.383490 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48da26e2-1645-4855-87b9-c050f5e22943 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:29.383778 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48da26e2-1645-4855-87b9-c050f5e22943 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:29.384058 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48da26e2-1645-4855-87b9-c050f5e22943 None admin] PATH_INFO: `/v3/projects/94ad8a7442da4d658f58d1543ae06493` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:29.386096 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48da26e2-1645-4855-87b9-c050f5e22943 None admin] RBAC: Authorizing `identity:get_project(project_id=94ad8a7442da4d658f58d1543ae06493)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:54:29.388071 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48da26e2-1645-4855-87b9-c050f5e22943 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:54:29.390156 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 115/233] 10.222.0.46 () {56 vars in 1186 bytes} [Thu May 26 14:54:29 2022] GET /identity/v3/projects/94ad8a7442da4d658f58d1543ae06493 => generated 302 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:54:29.395620 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c58f2b-aa4e-49f2-a495-aeedd7987470 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:29.457886 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:29.534049 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c58f2b-aa4e-49f2-a495-aeedd7987470 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:29.570603 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9847a76f-c7d3-468f-963f-28fc2bed1bcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:29.571508 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9847a76f-c7d3-468f-963f-28fc2bed1bcc None admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:29.571788 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9847a76f-c7d3-468f-963f-28fc2bed1bcc None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:29.572063 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9847a76f-c7d3-468f-963f-28fc2bed1bcc None admin] PATH_INFO: `/v3/projects/94ad8a7442da4d658f58d1543ae06493/users/bde8615a74bc494d9b747dffaa2f3b94/roles/160115b2143c4b389d39cd93f43e1d01` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:29.574901 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9847a76f-c7d3-468f-963f-28fc2bed1bcc None admin] RBAC: Authorizing `identity:create_grant(project_id=94ad8a7442da4d658f58d1543ae06493, user_id=bde8615a74bc494d9b747dffaa2f3b94, role_id=160115b2143c4b389d39cd93f43e1d01)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:54:29.576560 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9847a76f-c7d3-468f-963f-28fc2bed1bcc None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:54:29.597143 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 119/234] 10.222.0.46 () {58 vars in 1439 bytes} [Thu May 26 14:54:29 2022] PUT /identity/v3/projects/94ad8a7442da4d658f58d1543ae06493/users/bde8615a74bc494d9b747dffaa2f3b94/roles/160115b2143c4b389d39cd93f43e1d01 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 14:54:30.151761 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2f3c40-b81f-4371-9be8-bd0f06ec71e6 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:30.152813 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2f3c40-b81f-4371-9be8-bd0f06ec71e6 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:30.153301 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2f3c40-b81f-4371-9be8-bd0f06ec71e6 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:30.155156 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 116/235] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 14:54:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 14:54:30.161242 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95773343-4d05-4d44-919f-483046a389f3 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:30.161724 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95773343-4d05-4d44-919f-483046a389f3 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:30.162165 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95773343-4d05-4d44-919f-483046a389f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:30.191378 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-95773343-4d05-4d44-919f-483046a389f3 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:54:30.193115 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-95773343-4d05-4d44-919f-483046a389f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:54:30.325378 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 120/236] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 14:54:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 14:54:30.362358 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2f3c40-b81f-4371-9be8-bd0f06ec71e6 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:30.395708 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:30.492414 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2f3c40-b81f-4371-9be8-bd0f06ec71e6 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:30.533609 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e74711a6-edb9-4e21-918b-d0bee465f7b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:30.534968 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74711a6-edb9-4e21-918b-d0bee465f7b4 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:30.535266 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74711a6-edb9-4e21-918b-d0bee465f7b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:30.535544 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e74711a6-edb9-4e21-918b-d0bee465f7b4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:30.536795 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74711a6-edb9-4e21-918b-d0bee465f7b4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:54:30.538292 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e74711a6-edb9-4e21-918b-d0bee465f7b4 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:54:30.540788 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 117/237] 10.222.0.46 () {56 vars in 1108 bytes} [Thu May 26 14:54:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:54:30.547196 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95773343-4d05-4d44-919f-483046a389f3 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:30.578570 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:30.580331 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741238c0-e064-4088-b0e2-e65b9e932865 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:30.580865 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741238c0-e064-4088-b0e2-e65b9e932865 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:30.581508 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741238c0-e064-4088-b0e2-e65b9e932865 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:30.583540 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 118/238] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 14:54:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 14:54:30.589320 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb9e80f-ecbd-48f3-b930-ff85ea53c142 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:30.590478 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb9e80f-ecbd-48f3-b930-ff85ea53c142 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:30.590959 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb9e80f-ecbd-48f3-b930-ff85ea53c142 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:30.626881 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-dbb9e80f-ecbd-48f3-b930-ff85ea53c142 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:54:30.629505 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-dbb9e80f-ecbd-48f3-b930-ff85ea53c142 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:54:30.743007 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95773343-4d05-4d44-919f-483046a389f3 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:30.763631 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 119/239] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 14:54:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 14:54:30.794597 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb9e80f-ecbd-48f3-b930-ff85ea53c142 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:30.801936 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b78bbf-30a1-4568-83f8-6e491c4aca8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:30.802652 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b78bbf-30a1-4568-83f8-6e491c4aca8e None admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:30.802994 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b78bbf-30a1-4568-83f8-6e491c4aca8e None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:30.803277 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b78bbf-30a1-4568-83f8-6e491c4aca8e None admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:30.804162 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b78bbf-30a1-4568-83f8-6e491c4aca8e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:54:30.805883 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b78bbf-30a1-4568-83f8-6e491c4aca8e None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:54:30.846580 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 121/240] 10.222.0.46 () {60 vars in 1132 bytes} [Thu May 26 14:54:30 2022] POST /identity/v3/users => generated 293 bytes in 300 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 14:54:30.857911 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:30.967773 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb9e80f-ecbd-48f3-b930-ff85ea53c142 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:31.019292 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cbb04a-c09e-41d5-90ec-e2c74170692e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:31.020399 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cbb04a-c09e-41d5-90ec-e2c74170692e None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:31.020857 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cbb04a-c09e-41d5-90ec-e2c74170692e None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:31.021320 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cbb04a-c09e-41d5-90ec-e2c74170692e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:31.042481 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32cbb04a-c09e-41d5-90ec-e2c74170692e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 14:54:31.050981 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-32cbb04a-c09e-41d5-90ec-e2c74170692e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 26 14:54:31.050981 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:54:31.050981 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:54:31.050981 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:54:31.050981 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:54:31.050981 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:54:31.050981 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:54:31.050981 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:54:31.050981 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:54:31.050981 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:54:31.050981 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:54:31.050981 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:54:31.050981 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 14:54:31.050981 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 14:54:31.050981 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 14:54:31.050981 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 14:54:31.051926 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 14:54:31.051926 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 14:54:31.051926 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 14:54:31.051926 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 14:54:31.051926 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:54:31.051926 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:54:31.051926 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:54:31.051926 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:54:31.051926 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:54:31.051926 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:54:31.051926 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:54:31.051926 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 26 14:54:31.051926 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:54:31.051926 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 26 14:54:31.051926 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:54:31.051926 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:54:31.052698 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:54:31.052698 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 26 14:54:31.052698 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:54:31.052698 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 26 14:54:31.052698 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 14:54:31.052698 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 14:54:31.052698 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 14:54:31.052698 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 14:54:31.052698 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 14:54:31.052698 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 14:54:31.052698 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 26 14:54:31.052698 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 26 14:54:31.054909 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 120/241] 10.222.0.46 () {56 vars in 1099 bytes} [Thu May 26 14:54:30 2022] GET /identity/v3/roles/reader => generated 84 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 14:54:31.061434 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b78bbf-30a1-4568-83f8-6e491c4aca8e None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:31.147990 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec199ecb-b832-47d3-98e6-e28a941922d1 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:31.148286 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec199ecb-b832-47d3-98e6-e28a941922d1 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:31.148571 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec199ecb-b832-47d3-98e6-e28a941922d1 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:31.149992 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 121/242] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 14:54:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 14:54:31.153773 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:31.155453 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67ffda3-3cfa-42e9-90e1-c5fb949bea3e None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:31.155889 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67ffda3-3cfa-42e9-90e1-c5fb949bea3e None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:31.156023 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67ffda3-3cfa-42e9-90e1-c5fb949bea3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:31.182191 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-e67ffda3-3cfa-42e9-90e1-c5fb949bea3e None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:54:31.182662 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b78bbf-30a1-4568-83f8-6e491c4aca8e None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:31.185360 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-e67ffda3-3cfa-42e9-90e1-c5fb949bea3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:54:31.225684 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4813afd8-cf7b-4526-b7c9-258bc443d5fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:31.226619 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4813afd8-cf7b-4526-b7c9-258bc443d5fb None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:31.227075 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4813afd8-cf7b-4526-b7c9-258bc443d5fb None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:31.227493 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4813afd8-cf7b-4526-b7c9-258bc443d5fb None admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:31.228423 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4813afd8-cf7b-4526-b7c9-258bc443d5fb None admin] RBAC: Adding query filter params (name=reader) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:54:31.228876 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4813afd8-cf7b-4526-b7c9-258bc443d5fb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:54:31.230631 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4813afd8-cf7b-4526-b7c9-258bc443d5fb None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:54:31.240396 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 122/243] 10.222.0.46 () {56 vars in 1101 bytes} [Thu May 26 14:54:31 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:54:31.246849 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4813afd8-cf7b-4526-b7c9-258bc443d5fb None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:31.266370 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 122/244] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 14:54:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 14:54:31.296497 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67ffda3-3cfa-42e9-90e1-c5fb949bea3e None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:31.327270 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:31.394064 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4813afd8-cf7b-4526-b7c9-258bc443d5fb None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:31.454864 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:31.467570 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67ffda3-3cfa-42e9-90e1-c5fb949bea3e None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:31.468363 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ef7bfa-c9f9-4b95-8f22-aaebee2537f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:31.473491 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ef7bfa-c9f9-4b95-8f22-aaebee2537f6 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:31.473491 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ef7bfa-c9f9-4b95-8f22-aaebee2537f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:31.473491 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ef7bfa-c9f9-4b95-8f22-aaebee2537f6 None admin] PATH_INFO: `/v3/users/b255355211fa4d81be18915a3ff1bef7` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:31.499375 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ef7bfa-c9f9-4b95-8f22-aaebee2537f6 None admin] RBAC: Authorizing `identity:get_user(user_id=b255355211fa4d81be18915a3ff1bef7)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:54:31.501479 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ef7bfa-c9f9-4b95-8f22-aaebee2537f6 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:54:31.503835 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 123/245] 10.222.0.46 () {56 vars in 1177 bytes} [Thu May 26 14:54:31 2022] GET /identity/v3/users/b255355211fa4d81be18915a3ff1bef7 => generated 297 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 14:54:31.514153 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ef7bfa-c9f9-4b95-8f22-aaebee2537f6 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:31.526505 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe58d7b-a63b-4763-9e35-6655e11802a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:31.527887 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe58d7b-a63b-4763-9e35-6655e11802a5 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:31.528171 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe58d7b-a63b-4763-9e35-6655e11802a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:31.528445 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe58d7b-a63b-4763-9e35-6655e11802a5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:31.540295 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe58d7b-a63b-4763-9e35-6655e11802a5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 14:54:31.553896 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-9fe58d7b-a63b-4763-9e35-6655e11802a5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 26 14:54:31.553896 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:54:31.553896 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:54:31.553896 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:54:31.553896 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:54:31.553896 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:54:31.553896 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:54:31.553896 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:54:31.553896 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:54:31.553896 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:54:31.553896 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:54:31.553896 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:54:31.553896 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 14:54:31.553896 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 14:54:31.553896 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 14:54:31.553896 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 14:54:31.554821 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 14:54:31.554821 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 14:54:31.554821 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 14:54:31.554821 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 14:54:31.554821 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:54:31.554821 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:54:31.554821 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:54:31.554821 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:54:31.554821 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:54:31.554821 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:54:31.554821 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:54:31.554821 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 26 14:54:31.554821 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:54:31.554821 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 26 14:54:31.554821 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:54:31.554821 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:54:31.555633 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:54:31.555633 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 26 14:54:31.555633 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:54:31.555633 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 26 14:54:31.555633 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 14:54:31.555633 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 14:54:31.555633 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 14:54:31.555633 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 14:54:31.555633 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 14:54:31.555633 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 14:54:31.555633 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 26 14:54:31.555633 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 26 14:54:31.555633 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 123/246] 10.222.0.46 () {56 vars in 1099 bytes} [Thu May 26 14:54:31 2022] GET /identity/v3/roles/member => generated 84 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:54:31.561305 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe58d7b-a63b-4763-9e35-6655e11802a5 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:31.644213 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ef7bfa-c9f9-4b95-8f22-aaebee2537f6 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:31.681755 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:31.684469 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a391b1-8118-49dd-8972-194edd6e252f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:31.685146 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a391b1-8118-49dd-8972-194edd6e252f None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:31.685574 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a391b1-8118-49dd-8972-194edd6e252f None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:31.685865 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a391b1-8118-49dd-8972-194edd6e252f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:31.715139 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe58d7b-a63b-4763-9e35-6655e11802a5 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:31.752482 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:31.765553 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a391b1-8118-49dd-8972-194edd6e252f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:54:31.767063 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a391b1-8118-49dd-8972-194edd6e252f None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:54:31.773781 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e79775c-1d75-4a96-899e-7e4913707339 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:31.774755 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e79775c-1d75-4a96-899e-7e4913707339 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:31.775311 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e79775c-1d75-4a96-899e-7e4913707339 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:31.775740 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e79775c-1d75-4a96-899e-7e4913707339 None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:31.776855 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e79775c-1d75-4a96-899e-7e4913707339 None admin] RBAC: Adding query filter params (name=member) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:54:31.777493 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e79775c-1d75-4a96-899e-7e4913707339 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:54:31.779713 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e79775c-1d75-4a96-899e-7e4913707339 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:54:31.802043 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 124/247] 10.222.0.46 () {56 vars in 1101 bytes} [Thu May 26 14:54:31 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 14:54:31.804037 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e79775c-1d75-4a96-899e-7e4913707339 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:31.878962 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 124/248] 10.222.0.46 () {58 vars in 1282 bytes} [Thu May 26 14:54:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 14:54:31.880325 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db11485a-3823-4820-823b-846d7e18a3e4 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:31.880883 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db11485a-3823-4820-823b-846d7e18a3e4 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:31.881269 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db11485a-3823-4820-823b-846d7e18a3e4 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:31.882953 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 125/249] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 14:54:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 14:54:31.885836 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db11485a-3823-4820-823b-846d7e18a3e4 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:31.960541 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e79775c-1d75-4a96-899e-7e4913707339 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:31.994492 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:32.002145 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61dbb194-2a0a-465d-a6a6-85c6f15d13fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:32.003141 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dbb194-2a0a-465d-a6a6-85c6f15d13fc None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:32.003434 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dbb194-2a0a-465d-a6a6-85c6f15d13fc None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:32.003716 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dbb194-2a0a-465d-a6a6-85c6f15d13fc None admin] PATH_INFO: `/v3/users/4bddaf6a73f244baa66f51396d56e01f` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:32.004937 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61dbb194-2a0a-465d-a6a6-85c6f15d13fc None admin] RBAC: Authorizing `identity:get_user(user_id=4bddaf6a73f244baa66f51396d56e01f)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:54:32.007372 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61dbb194-2a0a-465d-a6a6-85c6f15d13fc None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:54:32.007744 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db11485a-3823-4820-823b-846d7e18a3e4 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:32.009435 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 125/250] 10.222.0.46 () {56 vars in 1177 bytes} [Thu May 26 14:54:31 2022] GET /identity/v3/users/4bddaf6a73f244baa66f51396d56e01f => generated 297 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:54:32.010771 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e8d218-7d7b-4992-a22d-fd5f5f694a22 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:32.011066 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e8d218-7d7b-4992-a22d-fd5f5f694a22 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:32.011931 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e8d218-7d7b-4992-a22d-fd5f5f694a22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:32.029719 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-60e8d218-7d7b-4992-a22d-fd5f5f694a22 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:54:32.031028 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-60e8d218-7d7b-4992-a22d-fd5f5f694a22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:54:32.034157 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:32.053504 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76449cf7-ef58-4300-b82a-8cfdba8db23b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:32.053504 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76449cf7-ef58-4300-b82a-8cfdba8db23b None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:32.053504 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76449cf7-ef58-4300-b82a-8cfdba8db23b None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:32.053504 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76449cf7-ef58-4300-b82a-8cfdba8db23b None admin] PATH_INFO: `/v3/projects/94ad8a7442da4d658f58d1543ae06493` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:32.053955 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76449cf7-ef58-4300-b82a-8cfdba8db23b None admin] RBAC: Authorizing `identity:get_project(project_id=94ad8a7442da4d658f58d1543ae06493)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:54:32.055009 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76449cf7-ef58-4300-b82a-8cfdba8db23b None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:54:32.057232 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 126/251] 10.222.0.46 () {56 vars in 1186 bytes} [Thu May 26 14:54:31 2022] GET /identity/v3/projects/94ad8a7442da4d658f58d1543ae06493 => generated 302 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 14:54:32.058897 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76449cf7-ef58-4300-b82a-8cfdba8db23b None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:32.165707 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 126/252] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 14:54:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 14:54:32.166428 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e8d218-7d7b-4992-a22d-fd5f5f694a22 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:32.168918 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76449cf7-ef58-4300-b82a-8cfdba8db23b None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:32.206010 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80322e43-56af-4411-a7e4-2243168a5ca0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:32.207154 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80322e43-56af-4411-a7e4-2243168a5ca0 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:32.207577 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80322e43-56af-4411-a7e4-2243168a5ca0 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:32.208091 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80322e43-56af-4411-a7e4-2243168a5ca0 None admin] PATH_INFO: `/v3/projects/94ad8a7442da4d658f58d1543ae06493` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:32.209745 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80322e43-56af-4411-a7e4-2243168a5ca0 None admin] RBAC: Authorizing `identity:get_project(project_id=94ad8a7442da4d658f58d1543ae06493)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:54:32.211367 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80322e43-56af-4411-a7e4-2243168a5ca0 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:54:32.213276 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 127/253] 10.222.0.46 () {56 vars in 1186 bytes} [Thu May 26 14:54:32 2022] GET /identity/v3/projects/94ad8a7442da4d658f58d1543ae06493 => generated 302 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:54:32.214185 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80322e43-56af-4411-a7e4-2243168a5ca0 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:32.249689 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:32.329213 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80322e43-56af-4411-a7e4-2243168a5ca0 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:32.367389 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8801a9cc-b643-4ce5-8ece-2f5b58c076ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:32.368556 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8801a9cc-b643-4ce5-8ece-2f5b58c076ea None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:32.369078 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8801a9cc-b643-4ce5-8ece-2f5b58c076ea None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:32.369726 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8801a9cc-b643-4ce5-8ece-2f5b58c076ea None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:32.385481 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e8d218-7d7b-4992-a22d-fd5f5f694a22 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:32.394414 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8801a9cc-b643-4ce5-8ece-2f5b58c076ea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 14:54:32.411253 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-8801a9cc-b643-4ce5-8ece-2f5b58c076ea None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 14:54:32.411253 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:54:32.411253 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:54:32.411253 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:54:32.411253 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:54:32.411253 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:54:32.411253 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:54:32.411253 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:54:32.411253 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:54:32.411253 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:54:32.411253 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:54:32.411253 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:54:32.411253 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 14:54:32.411253 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 14:54:32.411253 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 14:54:32.411253 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 14:54:32.412252 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 14:54:32.412252 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 14:54:32.412252 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 14:54:32.412252 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 14:54:32.412252 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:54:32.412252 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:54:32.412252 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:54:32.412252 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:54:32.412252 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:54:32.412252 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:54:32.412252 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:54:32.412252 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 26 14:54:32.412252 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:54:32.412252 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 26 14:54:32.412252 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:54:32.412252 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:54:32.413056 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:54:32.413056 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 26 14:54:32.413056 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:54:32.413056 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 26 14:54:32.413056 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 14:54:32.413056 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 14:54:32.413056 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 14:54:32.413056 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 14:54:32.413056 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 14:54:32.413056 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 14:54:32.413056 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 14:54:32.413056 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 26 14:54:32.418065 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 128/254] 10.222.0.46 () {56 vars in 1096 bytes} [Thu May 26 14:54:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:54:32.419260 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8801a9cc-b643-4ce5-8ece-2f5b58c076ea None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:32.468172 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80019f7-ece7-4f67-8894-1e11d0d1d617 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:32.469273 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80019f7-ece7-4f67-8894-1e11d0d1d617 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:32.469850 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80019f7-ece7-4f67-8894-1e11d0d1d617 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:32.470320 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80019f7-ece7-4f67-8894-1e11d0d1d617 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:32.471415 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80019f7-ece7-4f67-8894-1e11d0d1d617 None admin] RBAC: Adding query filter params (role.id=adb2011fe8f74ecbbb94465253e76c1c, scope.project.id=94ad8a7442da4d658f58d1543ae06493, user.id=b255355211fa4d81be18915a3ff1bef7) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:54:32.471924 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80019f7-ece7-4f67-8894-1e11d0d1d617 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:54:32.475649 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a80019f7-ece7-4f67-8894-1e11d0d1d617 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:54:32.487839 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 127/255] 10.222.0.46 () {56 vars in 1374 bytes} [Thu May 26 14:54:32 2022] GET /identity/v3/role_assignments?user.id=b255355211fa4d81be18915a3ff1bef7&scope.project.id=94ad8a7442da4d658f58d1543ae06493&role.id=adb2011fe8f74ecbbb94465253e76c1c => generated 259 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:54:32.491684 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80019f7-ece7-4f67-8894-1e11d0d1d617 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:32.502366 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:32.582409 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:32.600227 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8801a9cc-b643-4ce5-8ece-2f5b58c076ea None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:32.640276 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592922ad-e97a-4948-995d-d47a852fcc4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:32.641576 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592922ad-e97a-4948-995d-d47a852fcc4a None admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:32.642065 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592922ad-e97a-4948-995d-d47a852fcc4a None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:32.642555 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592922ad-e97a-4948-995d-d47a852fcc4a None admin] PATH_INFO: `/v3/projects/94ad8a7442da4d658f58d1543ae06493/users/4bddaf6a73f244baa66f51396d56e01f/roles/d4088d1845a24fbd9bd8975ce0b1d3d0` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:32.644966 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592922ad-e97a-4948-995d-d47a852fcc4a None admin] RBAC: Authorizing `identity:create_grant(project_id=94ad8a7442da4d658f58d1543ae06493, user_id=4bddaf6a73f244baa66f51396d56e01f, role_id=d4088d1845a24fbd9bd8975ce0b1d3d0)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:54:32.647587 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592922ad-e97a-4948-995d-d47a852fcc4a None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:54:32.670968 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 129/256] 10.222.0.46 () {58 vars in 1439 bytes} [Thu May 26 14:54:32 2022] PUT /identity/v3/projects/94ad8a7442da4d658f58d1543ae06493/users/4bddaf6a73f244baa66f51396d56e01f/roles/d4088d1845a24fbd9bd8975ce0b1d3d0 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 14:54:32.709287 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80019f7-ece7-4f67-8894-1e11d0d1d617 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:32.766964 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b982cec-d977-4a16-a040-8024d545ce00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:32.767665 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b982cec-d977-4a16-a040-8024d545ce00 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:32.767955 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b982cec-d977-4a16-a040-8024d545ce00 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:32.768238 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b982cec-d977-4a16-a040-8024d545ce00 None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:32.769032 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b982cec-d977-4a16-a040-8024d545ce00 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:54:32.769346 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b982cec-d977-4a16-a040-8024d545ce00 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:54:32.770847 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b982cec-d977-4a16-a040-8024d545ce00 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:54:32.786408 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 128/257] 10.222.0.46 () {56 vars in 1099 bytes} [Thu May 26 14:54:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:54:32.792006 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592922ad-e97a-4948-995d-d47a852fcc4a None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:32.934071 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592922ad-e97a-4948-995d-d47a852fcc4a None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:32.970622 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7b60de-e692-4978-870a-ef6583a98317 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:32.971881 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7b60de-e692-4978-870a-ef6583a98317 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:32.972358 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7b60de-e692-4978-870a-ef6583a98317 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:32.972825 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7b60de-e692-4978-870a-ef6583a98317 None admin] PATH_INFO: `/v3/users/bde8615a74bc494d9b747dffaa2f3b94` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:32.974436 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7b60de-e692-4978-870a-ef6583a98317 None admin] RBAC: Authorizing `identity:get_user(user_id=bde8615a74bc494d9b747dffaa2f3b94)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:54:32.976107 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7b60de-e692-4978-870a-ef6583a98317 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:54:32.978149 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 130/258] 10.222.0.46 () {56 vars in 1177 bytes} [Thu May 26 14:54:32 2022] GET /identity/v3/users/bde8615a74bc494d9b747dffaa2f3b94 => generated 283 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:54:32.984574 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b982cec-d977-4a16-a040-8024d545ce00 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:33.104218 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:33.150218 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b982cec-d977-4a16-a040-8024d545ce00 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:33.198460 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3734ff6-a518-42e4-bfb2-79bc052b8474 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:33.199108 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3734ff6-a518-42e4-bfb2-79bc052b8474 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:33.199356 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3734ff6-a518-42e4-bfb2-79bc052b8474 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:33.199632 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3734ff6-a518-42e4-bfb2-79bc052b8474 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:33.290300 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e2ee5c-28d9-4830-a1e9-3de197895761 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:33.290948 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e2ee5c-28d9-4830-a1e9-3de197895761 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:33.291448 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e2ee5c-28d9-4830-a1e9-3de197895761 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:33.293893 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 131/259] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 14:54:33 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 14:54:33.301489 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3734ff6-a518-42e4-bfb2-79bc052b8474 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:54:33.301489 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3734ff6-a518-42e4-bfb2-79bc052b8474 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:54:33.331402 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72898be4-9b8e-4a8f-9dbd-375c9ea3cf00 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:33.332004 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72898be4-9b8e-4a8f-9dbd-375c9ea3cf00 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:33.332926 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72898be4-9b8e-4a8f-9dbd-375c9ea3cf00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:33.394073 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-72898be4-9b8e-4a8f-9dbd-375c9ea3cf00 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:54:33.395904 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-72898be4-9b8e-4a8f-9dbd-375c9ea3cf00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:54:33.422741 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:33.455482 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:33.506289 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 132/260] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 14:54:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 14:54:33.519667 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 129/261] 10.222.0.46 () {58 vars in 1282 bytes} [Thu May 26 14:54:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 535 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 14:54:33.525349 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72898be4-9b8e-4a8f-9dbd-375c9ea3cf00 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:33.553574 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3734ff6-a518-42e4-bfb2-79bc052b8474 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:33.640757 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72898be4-9b8e-4a8f-9dbd-375c9ea3cf00 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:33.678605 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feab8f8e-ae3b-42d4-802b-c36b11623e4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:33.679480 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feab8f8e-ae3b-42d4-802b-c36b11623e4e None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:33.679761 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feab8f8e-ae3b-42d4-802b-c36b11623e4e None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:33.680035 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feab8f8e-ae3b-42d4-802b-c36b11623e4e None admin] PATH_INFO: `/v3/projects/94ad8a7442da4d658f58d1543ae06493` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:33.682174 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feab8f8e-ae3b-42d4-802b-c36b11623e4e None admin] RBAC: Authorizing `identity:get_project(project_id=94ad8a7442da4d658f58d1543ae06493)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:54:33.683681 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feab8f8e-ae3b-42d4-802b-c36b11623e4e None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:54:33.685404 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 133/262] 10.222.0.46 () {56 vars in 1186 bytes} [Thu May 26 14:54:33 2022] GET /identity/v3/projects/94ad8a7442da4d658f58d1543ae06493 => generated 302 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:54:33.692663 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feab8f8e-ae3b-42d4-802b-c36b11623e4e None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:33.725911 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:33.805629 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3734ff6-a518-42e4-bfb2-79bc052b8474 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:33.888258 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feab8f8e-ae3b-42d4-802b-c36b11623e4e None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:33.891064 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2ae458-7799-4011-ba49-be49b8384cca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:33.891889 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2ae458-7799-4011-ba49-be49b8384cca None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:33.892165 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2ae458-7799-4011-ba49-be49b8384cca None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:33.892434 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2ae458-7799-4011-ba49-be49b8384cca None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:33.925713 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2ae458-7799-4011-ba49-be49b8384cca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 14:54:33.935836 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-4d2ae458-7799-4011-ba49-be49b8384cca None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 14:54:33.935836 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:54:33.935836 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:54:33.935836 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:54:33.935836 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:54:33.935836 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:54:33.935836 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:54:33.935836 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:54:33.935836 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:54:33.935836 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:54:33.935836 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:54:33.935836 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:54:33.935836 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 14:54:33.935836 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 14:54:33.935836 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 14:54:33.935836 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 14:54:33.936787 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 14:54:33.936787 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 14:54:33.936787 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 14:54:33.936787 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 14:54:33.936787 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:54:33.936787 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:54:33.936787 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:54:33.936787 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:54:33.936787 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:54:33.936787 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:54:33.936787 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:54:33.936787 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 26 14:54:33.936787 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:54:33.936787 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 26 14:54:33.936787 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:54:33.936787 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:54:33.938102 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:54:33.938102 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 26 14:54:33.938102 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:54:33.938102 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 26 14:54:33.938102 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 14:54:33.938102 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 14:54:33.938102 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 14:54:33.938102 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 14:54:33.938102 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 14:54:33.938102 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 14:54:33.938102 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 14:54:33.938102 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 26 14:54:33.938102 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 130/263] 10.222.0.46 () {56 vars in 1114 bytes} [Thu May 26 14:54:33 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 384 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 14:54:33.950695 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2ae458-7799-4011-ba49-be49b8384cca None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:33.977897 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45625e53-902c-4f75-a812-3495fb7f8d72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:33.978613 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45625e53-902c-4f75-a812-3495fb7f8d72 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:33.980958 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45625e53-902c-4f75-a812-3495fb7f8d72 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:33.980958 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45625e53-902c-4f75-a812-3495fb7f8d72 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:33.980958 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45625e53-902c-4f75-a812-3495fb7f8d72 None admin] RBAC: Adding query filter params (role.id=a2e551773cbd4df084020676865b8222, scope.project.id=94ad8a7442da4d658f58d1543ae06493, user.id=bde8615a74bc494d9b747dffaa2f3b94) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:54:33.980958 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45625e53-902c-4f75-a812-3495fb7f8d72 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:54:33.984342 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45625e53-902c-4f75-a812-3495fb7f8d72 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:54:34.000450 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 134/264] 10.222.0.46 () {56 vars in 1374 bytes} [Thu May 26 14:54:33 2022] GET /identity/v3/role_assignments?user.id=bde8615a74bc494d9b747dffaa2f3b94&scope.project.id=94ad8a7442da4d658f58d1543ae06493&role.id=a2e551773cbd4df084020676865b8222 => generated 608 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:54:34.010680 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:34.155655 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2ae458-7799-4011-ba49-be49b8384cca None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:34.211531 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dad10ff-8f09-4de9-a151-f4d731a47c98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:34.212427 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dad10ff-8f09-4de9-a151-f4d731a47c98 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:34.212541 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dad10ff-8f09-4de9-a151-f4d731a47c98 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:34.212885 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dad10ff-8f09-4de9-a151-f4d731a47c98 None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:34.213530 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dad10ff-8f09-4de9-a151-f4d731a47c98 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:54:34.213841 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dad10ff-8f09-4de9-a151-f4d731a47c98 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:54:34.215302 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dad10ff-8f09-4de9-a151-f4d731a47c98 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:54:34.232099 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 131/265] 10.222.0.46 () {56 vars in 1111 bytes} [Thu May 26 14:54:33 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:54:34.240609 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45625e53-902c-4f75-a812-3495fb7f8d72 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:34.354020 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:34.379240 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45625e53-902c-4f75-a812-3495fb7f8d72 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:34.416423 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94b3da5-3101-43e5-99fb-c574fbe01f93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:34.417316 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94b3da5-3101-43e5-99fb-c574fbe01f93 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:34.418172 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94b3da5-3101-43e5-99fb-c574fbe01f93 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:34.418451 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94b3da5-3101-43e5-99fb-c574fbe01f93 None admin] PATH_INFO: `/v3/users/bde8615a74bc494d9b747dffaa2f3b94` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:34.419688 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94b3da5-3101-43e5-99fb-c574fbe01f93 None admin] RBAC: Authorizing `identity:get_user(user_id=bde8615a74bc494d9b747dffaa2f3b94)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:54:34.421178 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94b3da5-3101-43e5-99fb-c574fbe01f93 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:54:34.423248 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 135/266] 10.222.0.46 () {56 vars in 1177 bytes} [Thu May 26 14:54:34 2022] GET /identity/v3/users/bde8615a74bc494d9b747dffaa2f3b94 => generated 283 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:54:34.432688 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dad10ff-8f09-4de9-a151-f4d731a47c98 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:34.568434 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:34.586373 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dad10ff-8f09-4de9-a151-f4d731a47c98 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:34.653204 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58ef36e-ca9b-4af4-9a5b-c8f13f0cecfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:34.654067 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58ef36e-ca9b-4af4-9a5b-c8f13f0cecfc None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:34.654360 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58ef36e-ca9b-4af4-9a5b-c8f13f0cecfc None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:34.654645 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58ef36e-ca9b-4af4-9a5b-c8f13f0cecfc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:34.769916 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58ef36e-ca9b-4af4-9a5b-c8f13f0cecfc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:54:34.772409 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58ef36e-ca9b-4af4-9a5b-c8f13f0cecfc None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:54:34.905958 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:34.924180 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 132/267] 10.222.0.46 () {58 vars in 1282 bytes} [Thu May 26 14:54:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 492 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 14:54:34.930317 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94b3da5-3101-43e5-99fb-c574fbe01f93 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:34.947873 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea21819-260b-496c-a14b-1aed1e342c57 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:34.948194 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea21819-260b-496c-a14b-1aed1e342c57 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:34.948445 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea21819-260b-496c-a14b-1aed1e342c57 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:34.950563 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 133/268] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 14:54:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 14:54:34.960314 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c303af-4144-462b-9a22-bbbcbdb67094 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:34.960606 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c303af-4144-462b-9a22-bbbcbdb67094 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:34.960888 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c303af-4144-462b-9a22-bbbcbdb67094 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:34.988405 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-06c303af-4144-462b-9a22-bbbcbdb67094 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:54:34.989689 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-06c303af-4144-462b-9a22-bbbcbdb67094 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:54:35.142445 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94b3da5-3101-43e5-99fb-c574fbe01f93 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:35.159531 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 134/269] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 14:54:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 14:54:35.162248 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bbd14d-3837-4c43-80dd-c8def1878559 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:35.163205 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bbd14d-3837-4c43-80dd-c8def1878559 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:35.163912 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bbd14d-3837-4c43-80dd-c8def1878559 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:35.173582 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 135/270] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 14:54:35 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 14:54:35.176681 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e888ab6-4668-4842-8195-6e2dd6cc6018 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:35.177506 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e888ab6-4668-4842-8195-6e2dd6cc6018 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:35.178248 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e888ab6-4668-4842-8195-6e2dd6cc6018 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:35.199678 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-5e888ab6-4668-4842-8195-6e2dd6cc6018 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:54:35.201272 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-5e888ab6-4668-4842-8195-6e2dd6cc6018 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:54:35.228393 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bcc338-5b8a-48ed-95cc-653457604e99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:35.230474 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bcc338-5b8a-48ed-95cc-653457604e99 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:35.231016 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bcc338-5b8a-48ed-95cc-653457604e99 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:35.231565 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bcc338-5b8a-48ed-95cc-653457604e99 None admin] PATH_INFO: `/v3/projects/94ad8a7442da4d658f58d1543ae06493` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:35.233583 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63bcc338-5b8a-48ed-95cc-653457604e99 None admin] RBAC: Authorizing `identity:get_project(project_id=94ad8a7442da4d658f58d1543ae06493)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:54:35.236144 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63bcc338-5b8a-48ed-95cc-653457604e99 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:54:35.239293 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 136/271] 10.222.0.46 () {56 vars in 1186 bytes} [Thu May 26 14:54:34 2022] GET /identity/v3/projects/94ad8a7442da4d658f58d1543ae06493 => generated 302 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:54:35.247457 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bcc338-5b8a-48ed-95cc-653457604e99 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:35.265900 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:35.300533 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 136/272] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 14:54:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 14:54:35.305817 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e888ab6-4668-4842-8195-6e2dd6cc6018 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:35.388011 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bcc338-5b8a-48ed-95cc-653457604e99 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:35.393848 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:35.430412 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834f7d5c-bc45-49cd-947f-b9eb608e6cc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:35.430792 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834f7d5c-bc45-49cd-947f-b9eb608e6cc7 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:35.431086 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834f7d5c-bc45-49cd-947f-b9eb608e6cc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:35.431431 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834f7d5c-bc45-49cd-947f-b9eb608e6cc7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:35.432696 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834f7d5c-bc45-49cd-947f-b9eb608e6cc7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:54:35.434365 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834f7d5c-bc45-49cd-947f-b9eb608e6cc7 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:54:35.436161 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 137/273] 10.222.0.46 () {56 vars in 1108 bytes} [Thu May 26 14:54:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:54:35.441473 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834f7d5c-bc45-49cd-947f-b9eb608e6cc7 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:35.461820 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e888ab6-4668-4842-8195-6e2dd6cc6018 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:35.512035 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a8c08b-544a-4623-9481-b41bdd036f08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:35.513086 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a8c08b-544a-4623-9481-b41bdd036f08 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:35.513595 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a8c08b-544a-4623-9481-b41bdd036f08 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:35.514066 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a8c08b-544a-4623-9481-b41bdd036f08 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:35.516216 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a8c08b-544a-4623-9481-b41bdd036f08 None admin] RBAC: Adding query filter params (role.id=160115b2143c4b389d39cd93f43e1d01, scope.project.id=94ad8a7442da4d658f58d1543ae06493, user.id=bde8615a74bc494d9b747dffaa2f3b94) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:54:35.516874 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a8c08b-544a-4623-9481-b41bdd036f08 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:54:35.519048 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a8c08b-544a-4623-9481-b41bdd036f08 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:54:35.528636 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 137/274] 10.222.0.46 () {56 vars in 1374 bytes} [Thu May 26 14:54:35 2022] GET /identity/v3/role_assignments?user.id=bde8615a74bc494d9b747dffaa2f3b94&scope.project.id=94ad8a7442da4d658f58d1543ae06493&role.id=160115b2143c4b389d39cd93f43e1d01 => generated 608 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:54:35.529807 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a8c08b-544a-4623-9481-b41bdd036f08 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:35.577473 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:35.645583 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834f7d5c-bc45-49cd-947f-b9eb608e6cc7 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:35.670509 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a8c08b-544a-4623-9481-b41bdd036f08 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:35.689422 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:35.710508 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fb1d1a-7cda-4e92-aa22-d0f9810e16c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:35.712024 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fb1d1a-7cda-4e92-aa22-d0f9810e16c1 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:35.713073 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fb1d1a-7cda-4e92-aa22-d0f9810e16c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:35.713822 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fb1d1a-7cda-4e92-aa22-d0f9810e16c1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:35.717881 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb2ccd1-0717-4e0a-827b-e093265773ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:35.718486 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb2ccd1-0717-4e0a-827b-e093265773ca None admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:35.718847 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb2ccd1-0717-4e0a-827b-e093265773ca None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:35.719205 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb2ccd1-0717-4e0a-827b-e093265773ca None admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:35.720013 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb2ccd1-0717-4e0a-827b-e093265773ca None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:54:35.721645 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb2ccd1-0717-4e0a-827b-e093265773ca None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:54:35.728359 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fb1d1a-7cda-4e92-aa22-d0f9810e16c1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 14:54:35.739157 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-b9fb1d1a-7cda-4e92-aa22-d0f9810e16c1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 26 14:54:35.739157 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:54:35.739157 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:54:35.739157 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:54:35.739157 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:54:35.739157 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:54:35.739157 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:54:35.739157 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:54:35.739157 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:54:35.739157 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:54:35.739157 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:54:35.739157 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:54:35.739157 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 14:54:35.739157 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 14:54:35.739157 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 14:54:35.739157 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 14:54:35.740093 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 14:54:35.740093 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 14:54:35.740093 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 14:54:35.740093 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 14:54:35.740093 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:54:35.740093 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:54:35.740093 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:54:35.740093 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:54:35.740093 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:54:35.740093 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:54:35.740093 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:54:35.740093 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 26 14:54:35.740093 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:54:35.740093 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 26 14:54:35.740093 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:54:35.740093 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:54:35.740868 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:54:35.740868 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 26 14:54:35.740868 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:54:35.740868 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 26 14:54:35.740868 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 14:54:35.740868 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 14:54:35.740868 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 14:54:35.740868 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 14:54:35.740868 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 14:54:35.740868 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 14:54:35.740868 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 26 14:54:35.740868 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 26 14:54:35.744093 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 138/275] 10.222.0.46 () {56 vars in 1099 bytes} [Thu May 26 14:54:35 2022] GET /identity/v3/roles/member => generated 84 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:54:35.751252 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6f78ce-56cc-42a5-92e6-0d5a0eedd235 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:35.751796 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6f78ce-56cc-42a5-92e6-0d5a0eedd235 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:35.752248 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6f78ce-56cc-42a5-92e6-0d5a0eedd235 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:35.752897 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 138/276] 10.222.0.46 () {60 vars in 1132 bytes} [Thu May 26 14:54:35 2022] POST /identity/v3/users => generated 293 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 14:54:35.754803 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 139/277] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 14:54:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 14:54:35.760443 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb2ccd1-0717-4e0a-827b-e093265773ca None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:35.760783 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799e7393-1237-4af4-a498-7ef55cbe2881 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:35.761074 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799e7393-1237-4af4-a498-7ef55cbe2881 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:35.762456 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799e7393-1237-4af4-a498-7ef55cbe2881 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:35.784840 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-799e7393-1237-4af4-a498-7ef55cbe2881 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:54:35.787211 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-799e7393-1237-4af4-a498-7ef55cbe2881 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:54:35.893030 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 140/278] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 14:54:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 14:54:35.894802 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f28137-02ad-4f5f-b863-cd1347ee7881 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:35.895271 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f28137-02ad-4f5f-b863-cd1347ee7881 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:35.895786 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f28137-02ad-4f5f-b863-cd1347ee7881 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:35.897334 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 141/279] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 14:54:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 14:54:35.904539 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9bfc90-6777-4628-9747-c6ec636c77ff None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:35.905117 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9bfc90-6777-4628-9747-c6ec636c77ff None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:35.905812 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9bfc90-6777-4628-9747-c6ec636c77ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:35.919485 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb2ccd1-0717-4e0a-827b-e093265773ca None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:35.926159 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-cc9bfc90-6777-4628-9747-c6ec636c77ff None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:54:35.928099 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-cc9bfc90-6777-4628-9747-c6ec636c77ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:54:35.963520 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757bb1e8-012b-4f9e-a882-d2c7a5db0a03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:35.964121 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757bb1e8-012b-4f9e-a882-d2c7a5db0a03 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:35.964438 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757bb1e8-012b-4f9e-a882-d2c7a5db0a03 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:35.964718 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757bb1e8-012b-4f9e-a882-d2c7a5db0a03 None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:35.965542 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757bb1e8-012b-4f9e-a882-d2c7a5db0a03 None admin] RBAC: Adding query filter params (name=member) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:54:35.965836 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757bb1e8-012b-4f9e-a882-d2c7a5db0a03 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:54:35.967452 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757bb1e8-012b-4f9e-a882-d2c7a5db0a03 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:54:35.976520 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 139/280] 10.222.0.46 () {56 vars in 1101 bytes} [Thu May 26 14:54:35 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:54:35.977378 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757bb1e8-012b-4f9e-a882-d2c7a5db0a03 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:36.006187 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:36.011796 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 142/281] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 14:54:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 14:54:36.012687 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9bfc90-6777-4628-9747-c6ec636c77ff None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:36.025855 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:36.118555 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757bb1e8-012b-4f9e-a882-d2c7a5db0a03 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:36.153897 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9bfc90-6777-4628-9747-c6ec636c77ff None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:36.160700 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6b5d9a-bc9c-4795-ba6f-3df0404ae5d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:36.161538 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6b5d9a-bc9c-4795-ba6f-3df0404ae5d1 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:36.161825 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6b5d9a-bc9c-4795-ba6f-3df0404ae5d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:36.162100 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6b5d9a-bc9c-4795-ba6f-3df0404ae5d1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:36.173698 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6b5d9a-bc9c-4795-ba6f-3df0404ae5d1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 14:54:36.181617 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-1f6b5d9a-bc9c-4795-ba6f-3df0404ae5d1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 26 14:54:36.181617 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:54:36.181617 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:54:36.181617 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:54:36.181617 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:54:36.181617 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:54:36.181617 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:54:36.181617 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:54:36.181617 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:54:36.181617 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:54:36.181617 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:54:36.181617 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:54:36.181617 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 14:54:36.181617 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 14:54:36.181617 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 14:54:36.181617 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 14:54:36.182531 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 14:54:36.182531 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 14:54:36.182531 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 14:54:36.182531 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 14:54:36.182531 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:54:36.182531 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:54:36.182531 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:54:36.182531 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:54:36.182531 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:54:36.182531 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:54:36.182531 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:54:36.182531 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 26 14:54:36.182531 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:54:36.182531 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 26 14:54:36.182531 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:54:36.182531 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:54:36.183913 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:54:36.183913 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 26 14:54:36.183913 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:54:36.183913 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 26 14:54:36.183913 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 14:54:36.183913 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 14:54:36.183913 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 14:54:36.183913 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 14:54:36.183913 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 14:54:36.183913 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 14:54:36.183913 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 26 14:54:36.183913 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 26 14:54:36.183913 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 140/282] 10.222.0.46 () {56 vars in 1099 bytes} [Thu May 26 14:54:35 2022] GET /identity/v3/roles/member => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:54:36.184584 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6b5d9a-bc9c-4795-ba6f-3df0404ae5d1 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:36.207701 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44887661-4fe1-4dfb-89af-e6fb0dc77b15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:36.208587 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44887661-4fe1-4dfb-89af-e6fb0dc77b15 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:36.208866 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44887661-4fe1-4dfb-89af-e6fb0dc77b15 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:36.209141 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44887661-4fe1-4dfb-89af-e6fb0dc77b15 None admin] PATH_INFO: `/v3/users/c3026d3cb3004d7abf348314511477bb` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:36.223235 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44887661-4fe1-4dfb-89af-e6fb0dc77b15 None admin] RBAC: Authorizing `identity:get_user(user_id=c3026d3cb3004d7abf348314511477bb)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:54:36.224632 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44887661-4fe1-4dfb-89af-e6fb0dc77b15 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:54:36.226482 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 143/283] 10.222.0.46 () {56 vars in 1177 bytes} [Thu May 26 14:54:36 2022] GET /identity/v3/users/c3026d3cb3004d7abf348314511477bb => generated 293 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:54:36.227251 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44887661-4fe1-4dfb-89af-e6fb0dc77b15 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:36.299146 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:36.323136 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:36.337999 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6b5d9a-bc9c-4795-ba6f-3df0404ae5d1 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:36.346136 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44887661-4fe1-4dfb-89af-e6fb0dc77b15 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:36.382456 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c511f3dc-a738-4800-ab6c-3c4bd5f75f11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:36.383288 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c511f3dc-a738-4800-ab6c-3c4bd5f75f11 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:36.383687 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c511f3dc-a738-4800-ab6c-3c4bd5f75f11 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:36.383857 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c50cec6-93c5-4053-8241-b8b6a75c511e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:36.384143 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c511f3dc-a738-4800-ab6c-3c4bd5f75f11 None admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:36.384938 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c50cec6-93c5-4053-8241-b8b6a75c511e None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:36.385049 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c511f3dc-a738-4800-ab6c-3c4bd5f75f11 None admin] RBAC: Adding query filter params (name=member) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:54:36.385441 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c511f3dc-a738-4800-ab6c-3c4bd5f75f11 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:54:36.385610 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c50cec6-93c5-4053-8241-b8b6a75c511e None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:36.386008 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c50cec6-93c5-4053-8241-b8b6a75c511e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:36.387142 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c511f3dc-a738-4800-ab6c-3c4bd5f75f11 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:54:36.400653 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 144/284] 10.222.0.46 () {56 vars in 1101 bytes} [Thu May 26 14:54:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:54:36.400653 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c511f3dc-a738-4800-ab6c-3c4bd5f75f11 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:36.404260 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c50cec6-93c5-4053-8241-b8b6a75c511e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 14:54:36.414633 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-1c50cec6-93c5-4053-8241-b8b6a75c511e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 26 14:54:36.414633 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:54:36.414633 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:54:36.414633 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:54:36.414633 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:54:36.414633 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:54:36.414633 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:54:36.414633 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:54:36.414633 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:54:36.414633 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:54:36.414633 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:54:36.414633 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:54:36.414633 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 14:54:36.414633 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 14:54:36.414633 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 14:54:36.414633 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 14:54:36.415551 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 14:54:36.415551 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 14:54:36.415551 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 14:54:36.415551 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 14:54:36.415551 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:54:36.415551 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:54:36.415551 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:54:36.415551 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:54:36.415551 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:54:36.415551 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:54:36.415551 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:54:36.415551 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 26 14:54:36.415551 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:54:36.415551 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 26 14:54:36.415551 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:54:36.415551 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:54:36.416332 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:54:36.416332 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 26 14:54:36.416332 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:54:36.416332 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 26 14:54:36.416332 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 14:54:36.416332 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 14:54:36.416332 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 14:54:36.416332 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 14:54:36.416332 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 14:54:36.416332 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 14:54:36.416332 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 26 14:54:36.416332 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 26 14:54:36.416332 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 141/285] 10.222.0.46 () {56 vars in 1099 bytes} [Thu May 26 14:54:36 2022] GET /identity/v3/roles/reader => generated 84 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:54:36.418113 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c50cec6-93c5-4053-8241-b8b6a75c511e None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:36.521101 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c50cec6-93c5-4053-8241-b8b6a75c511e None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:36.536990 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c511f3dc-a738-4800-ab6c-3c4bd5f75f11 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:36.555251 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4344b7b-8032-4f57-92c4-9c3792ba0a57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:36.556121 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4344b7b-8032-4f57-92c4-9c3792ba0a57 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:36.556405 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4344b7b-8032-4f57-92c4-9c3792ba0a57 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:36.556959 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4344b7b-8032-4f57-92c4-9c3792ba0a57 None admin] PATH_INFO: `/v3/users/4bddaf6a73f244baa66f51396d56e01f` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:36.557919 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4344b7b-8032-4f57-92c4-9c3792ba0a57 None admin] RBAC: Authorizing `identity:get_user(user_id=4bddaf6a73f244baa66f51396d56e01f)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:54:36.559389 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4344b7b-8032-4f57-92c4-9c3792ba0a57 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:54:36.565119 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:36.565568 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 142/286] 10.222.0.46 () {56 vars in 1177 bytes} [Thu May 26 14:54:36 2022] GET /identity/v3/users/4bddaf6a73f244baa66f51396d56e01f => generated 297 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:54:36.566878 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4344b7b-8032-4f57-92c4-9c3792ba0a57 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:36.584202 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587ea5a8-493d-4fb1-a10a-4f320b06e641 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:36.585441 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587ea5a8-493d-4fb1-a10a-4f320b06e641 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:36.586079 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587ea5a8-493d-4fb1-a10a-4f320b06e641 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:36.586574 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587ea5a8-493d-4fb1-a10a-4f320b06e641 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:36.587850 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587ea5a8-493d-4fb1-a10a-4f320b06e641 None admin] RBAC: Adding query filter params (role.id=d4088d1845a24fbd9bd8975ce0b1d3d0, user.id=c3026d3cb3004d7abf348314511477bb, scope.system=all) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:54:36.588358 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587ea5a8-493d-4fb1-a10a-4f320b06e641 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:54:36.591099 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-587ea5a8-493d-4fb1-a10a-4f320b06e641 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:54:36.600188 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:36.601554 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 145/287] 10.222.0.46 () {56 vars in 1308 bytes} [Thu May 26 14:54:36 2022] GET /identity/v3/role_assignments?user.id=c3026d3cb3004d7abf348314511477bb&scope.system=all&role.id=d4088d1845a24fbd9bd8975ce0b1d3d0 => generated 226 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:54:36.602717 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587ea5a8-493d-4fb1-a10a-4f320b06e641 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:36.764521 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4344b7b-8032-4f57-92c4-9c3792ba0a57 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:36.816237 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587ea5a8-493d-4fb1-a10a-4f320b06e641 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:36.839194 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e325917-2511-4205-a05a-1ba73fced760 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:36.840525 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e325917-2511-4205-a05a-1ba73fced760 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:36.840807 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e325917-2511-4205-a05a-1ba73fced760 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:36.841082 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e325917-2511-4205-a05a-1ba73fced760 None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:36.841864 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e325917-2511-4205-a05a-1ba73fced760 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:54:36.842171 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e325917-2511-4205-a05a-1ba73fced760 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:54:36.843563 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e325917-2511-4205-a05a-1ba73fced760 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:54:36.860385 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 143/288] 10.222.0.46 () {56 vars in 1101 bytes} [Thu May 26 14:54:36 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:54:36.866318 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb16d63-d20d-40e3-9301-2286ab3bc378 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:36.866523 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e325917-2511-4205-a05a-1ba73fced760 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:36.867011 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb16d63-d20d-40e3-9301-2286ab3bc378 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:36.867299 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb16d63-d20d-40e3-9301-2286ab3bc378 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:36.867575 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb16d63-d20d-40e3-9301-2286ab3bc378 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:36.944582 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:36.966979 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb16d63-d20d-40e3-9301-2286ab3bc378 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:54:36.969217 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb16d63-d20d-40e3-9301-2286ab3bc378 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:54:37.047659 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:37.092036 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 146/289] 10.222.0.46 () {58 vars in 1282 bytes} [Thu May 26 14:54:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 490 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 14:54:37.099250 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb16d63-d20d-40e3-9301-2286ab3bc378 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:37.120145 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e325917-2511-4205-a05a-1ba73fced760 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:37.162972 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac7faaa-fe7e-48aa-867b-3f6c01d79e2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:37.163860 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac7faaa-fe7e-48aa-867b-3f6c01d79e2b None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:37.164141 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac7faaa-fe7e-48aa-867b-3f6c01d79e2b None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:37.164421 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac7faaa-fe7e-48aa-867b-3f6c01d79e2b None admin] PATH_INFO: `/v3/users/b255355211fa4d81be18915a3ff1bef7` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:37.165698 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac7faaa-fe7e-48aa-867b-3f6c01d79e2b None admin] RBAC: Authorizing `identity:get_user(user_id=b255355211fa4d81be18915a3ff1bef7)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:54:37.167152 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac7faaa-fe7e-48aa-867b-3f6c01d79e2b None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:54:37.169955 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 144/290] 10.222.0.46 () {56 vars in 1177 bytes} [Thu May 26 14:54:36 2022] GET /identity/v3/users/b255355211fa4d81be18915a3ff1bef7 => generated 297 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 14:54:37.176057 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac7faaa-fe7e-48aa-867b-3f6c01d79e2b None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:37.317107 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:37.339997 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb16d63-d20d-40e3-9301-2286ab3bc378 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:37.354697 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac7faaa-fe7e-48aa-867b-3f6c01d79e2b None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:37.399944 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbca4c0-dbdc-47b5-8785-78b7dec69fad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:37.400829 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbca4c0-dbdc-47b5-8785-78b7dec69fad None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:37.401107 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbca4c0-dbdc-47b5-8785-78b7dec69fad None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:37.401382 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dbca4c0-dbdc-47b5-8785-78b7dec69fad None admin] PATH_INFO: `/v3/projects/94ad8a7442da4d658f58d1543ae06493` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:37.402735 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbca4c0-dbdc-47b5-8785-78b7dec69fad None admin] RBAC: Authorizing `identity:get_project(project_id=94ad8a7442da4d658f58d1543ae06493)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:54:37.405404 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dbca4c0-dbdc-47b5-8785-78b7dec69fad None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:54:37.407475 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 145/291] 10.222.0.46 () {56 vars in 1186 bytes} [Thu May 26 14:54:37 2022] GET /identity/v3/projects/94ad8a7442da4d658f58d1543ae06493 => generated 302 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:54:37.414132 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbca4c0-dbdc-47b5-8785-78b7dec69fad None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:37.426157 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9807f8-f16d-40ae-bef5-791f3db19770 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:37.427608 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9807f8-f16d-40ae-bef5-791f3db19770 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:37.428201 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9807f8-f16d-40ae-bef5-791f3db19770 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:37.428798 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9807f8-f16d-40ae-bef5-791f3db19770 None admin] PATH_INFO: `/v3/projects/94ad8a7442da4d658f58d1543ae06493` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:37.430465 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9807f8-f16d-40ae-bef5-791f3db19770 None admin] RBAC: Authorizing `identity:get_project(project_id=94ad8a7442da4d658f58d1543ae06493)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:54:37.432360 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb9807f8-f16d-40ae-bef5-791f3db19770 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:54:37.434934 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 147/292] 10.222.0.46 () {56 vars in 1186 bytes} [Thu May 26 14:54:37 2022] GET /identity/v3/projects/94ad8a7442da4d658f58d1543ae06493 => generated 302 bytes in 336 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:54:37.436231 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:37.453314 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9807f8-f16d-40ae-bef5-791f3db19770 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:37.544541 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dbca4c0-dbdc-47b5-8785-78b7dec69fad None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:37.585181 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20244183-2b55-4a7f-991a-5dbdd92e5ee9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:37.586865 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20244183-2b55-4a7f-991a-5dbdd92e5ee9 None admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:37.587295 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20244183-2b55-4a7f-991a-5dbdd92e5ee9 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:37.587734 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20244183-2b55-4a7f-991a-5dbdd92e5ee9 None admin] PATH_INFO: `/v3/projects/94ad8a7442da4d658f58d1543ae06493/users/b255355211fa4d81be18915a3ff1bef7/roles/adb2011fe8f74ecbbb94465253e76c1c` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:37.590866 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20244183-2b55-4a7f-991a-5dbdd92e5ee9 None admin] RBAC: Authorizing `identity:create_grant(project_id=94ad8a7442da4d658f58d1543ae06493, user_id=b255355211fa4d81be18915a3ff1bef7, role_id=adb2011fe8f74ecbbb94465253e76c1c)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:54:37.592820 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20244183-2b55-4a7f-991a-5dbdd92e5ee9 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:54:37.612888 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 146/293] 10.222.0.46 () {58 vars in 1439 bytes} [Thu May 26 14:54:37 2022] PUT /identity/v3/projects/94ad8a7442da4d658f58d1543ae06493/users/b255355211fa4d81be18915a3ff1bef7/roles/adb2011fe8f74ecbbb94465253e76c1c => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 14:54:37.707822 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9807f8-f16d-40ae-bef5-791f3db19770 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:37.767910 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df77840d-1ca6-4c34-8b67-ee10ad58103b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:37.768598 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df77840d-1ca6-4c34-8b67-ee10ad58103b None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:37.768882 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df77840d-1ca6-4c34-8b67-ee10ad58103b None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:37.769482 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df77840d-1ca6-4c34-8b67-ee10ad58103b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:37.770572 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df77840d-1ca6-4c34-8b67-ee10ad58103b None admin] RBAC: Adding query filter params (role.id=d4088d1845a24fbd9bd8975ce0b1d3d0, scope.project.id=94ad8a7442da4d658f58d1543ae06493, user.id=4bddaf6a73f244baa66f51396d56e01f) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:54:37.771197 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df77840d-1ca6-4c34-8b67-ee10ad58103b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:54:37.773352 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df77840d-1ca6-4c34-8b67-ee10ad58103b None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:54:37.791111 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 148/294] 10.222.0.46 () {56 vars in 1374 bytes} [Thu May 26 14:54:37 2022] GET /identity/v3/role_assignments?user.id=4bddaf6a73f244baa66f51396d56e01f&scope.project.id=94ad8a7442da4d658f58d1543ae06493&role.id=d4088d1845a24fbd9bd8975ce0b1d3d0 => generated 608 bytes in 338 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 14:54:38.370020 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506de605-664c-4a6b-abfd-c0709bf51a61 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:38.371630 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506de605-664c-4a6b-abfd-c0709bf51a61 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:38.372128 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506de605-664c-4a6b-abfd-c0709bf51a61 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:38.373994 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 147/295] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 14:54:38 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 14:54:38.379467 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6774c0-7c6b-4f40-8586-98a8b66c7b0a None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:38.380011 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6774c0-7c6b-4f40-8586-98a8b66c7b0a None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:38.380218 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6774c0-7c6b-4f40-8586-98a8b66c7b0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:38.403656 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-7e6774c0-7c6b-4f40-8586-98a8b66c7b0a None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:54:38.404885 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-7e6774c0-7c6b-4f40-8586-98a8b66c7b0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:54:38.421590 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:38.496970 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 149/296] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 14:54:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 14:54:38.532442 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506de605-664c-4a6b-abfd-c0709bf51a61 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:38.617829 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:38.646180 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506de605-664c-4a6b-abfd-c0709bf51a61 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:38.666760 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a09498c-51c6-4f9e-86db-d3851eeedfc6 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:38.667202 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a09498c-51c6-4f9e-86db-d3851eeedfc6 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:38.668169 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a09498c-51c6-4f9e-86db-d3851eeedfc6 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:38.669985 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 150/297] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 14:54:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 14:54:38.675198 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3426a15c-ade3-4158-ace2-ec87007e2a6d None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:38.675761 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3426a15c-ade3-4158-ace2-ec87007e2a6d None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:38.676138 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3426a15c-ade3-4158-ace2-ec87007e2a6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:38.686314 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f7a898-6246-43db-8633-36aa71592a9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:38.687345 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f7a898-6246-43db-8633-36aa71592a9b None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:38.687877 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f7a898-6246-43db-8633-36aa71592a9b None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:38.688308 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f7a898-6246-43db-8633-36aa71592a9b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:38.694016 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-3426a15c-ade3-4158-ace2-ec87007e2a6d None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:54:38.695372 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-3426a15c-ade3-4158-ace2-ec87007e2a6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:54:38.700987 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f7a898-6246-43db-8633-36aa71592a9b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 14:54:38.711305 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-e5f7a898-6246-43db-8633-36aa71592a9b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 26 14:54:38.711305 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:54:38.711305 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:54:38.711305 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:54:38.711305 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:54:38.711305 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:54:38.711305 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:54:38.711305 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:54:38.711305 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:54:38.711305 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:54:38.711305 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:54:38.711305 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:54:38.711305 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 14:54:38.711305 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 14:54:38.711305 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 14:54:38.711305 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 14:54:38.712436 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 14:54:38.712436 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 14:54:38.712436 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 14:54:38.712436 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 14:54:38.712436 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:54:38.712436 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:54:38.712436 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:54:38.712436 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:54:38.712436 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:54:38.712436 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:54:38.712436 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:54:38.712436 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 26 14:54:38.712436 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:54:38.712436 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 26 14:54:38.712436 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:54:38.712436 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:54:38.713404 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:54:38.713404 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 26 14:54:38.713404 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:54:38.713404 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 26 14:54:38.713404 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 14:54:38.713404 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 14:54:38.713404 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 14:54:38.713404 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 14:54:38.713404 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 14:54:38.713404 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 14:54:38.713404 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 26 14:54:38.713404 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 26 14:54:38.716366 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 148/298] 10.222.0.46 () {56 vars in 1099 bytes} [Thu May 26 14:54:38 2022] GET /identity/v3/roles/reader => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:54:38.722793 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f7a898-6246-43db-8633-36aa71592a9b None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:38.789927 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 151/299] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 14:54:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 14:54:38.805037 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3426a15c-ade3-4158-ace2-ec87007e2a6d None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:38.835050 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f7a898-6246-43db-8633-36aa71592a9b None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:38.871449 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30d6ab3-9ad6-4af5-a0ec-b94823301b28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:38.872397 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30d6ab3-9ad6-4af5-a0ec-b94823301b28 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:38.872829 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30d6ab3-9ad6-4af5-a0ec-b94823301b28 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:38.873295 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30d6ab3-9ad6-4af5-a0ec-b94823301b28 None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:38.874304 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30d6ab3-9ad6-4af5-a0ec-b94823301b28 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:54:38.874786 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30d6ab3-9ad6-4af5-a0ec-b94823301b28 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:54:38.876357 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30d6ab3-9ad6-4af5-a0ec-b94823301b28 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:54:38.879171 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:38.892758 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 149/300] 10.222.0.46 () {56 vars in 1101 bytes} [Thu May 26 14:54:38 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:54:38.894422 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30d6ab3-9ad6-4af5-a0ec-b94823301b28 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:38.894717 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:38.921576 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3426a15c-ade3-4158-ace2-ec87007e2a6d None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:38.961426 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8107668-7349-461f-ae00-b317fad03fcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:38.962402 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8107668-7349-461f-ae00-b317fad03fcc None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:38.962755 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8107668-7349-461f-ae00-b317fad03fcc None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:38.963136 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8107668-7349-461f-ae00-b317fad03fcc None admin] PATH_INFO: `/v3/domains/default` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:38.964597 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8107668-7349-461f-ae00-b317fad03fcc None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:54:38.966202 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8107668-7349-461f-ae00-b317fad03fcc None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:54:38.968033 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 152/301] 10.222.0.46 () {56 vars in 1108 bytes} [Thu May 26 14:54:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 14:54:38.973550 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8107668-7349-461f-ae00-b317fad03fcc None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:39.017106 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30d6ab3-9ad6-4af5-a0ec-b94823301b28 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:39.058809 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40664cb6-cf92-4184-a8b7-873b76ad5aa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:39.060075 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40664cb6-cf92-4184-a8b7-873b76ad5aa4 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:39.060539 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40664cb6-cf92-4184-a8b7-873b76ad5aa4 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:39.060986 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40664cb6-cf92-4184-a8b7-873b76ad5aa4 None admin] PATH_INFO: `/v3/users/454431a7cdaf4374a1090691a2b55560` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:39.076564 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40664cb6-cf92-4184-a8b7-873b76ad5aa4 None admin] RBAC: Authorizing `identity:get_user(user_id=454431a7cdaf4374a1090691a2b55560)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:54:39.078346 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40664cb6-cf92-4184-a8b7-873b76ad5aa4 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:54:39.080294 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 150/302] 10.222.0.46 () {56 vars in 1177 bytes} [Thu May 26 14:54:38 2022] GET /identity/v3/users/454431a7cdaf4374a1090691a2b55560 => generated 293 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:54:39.086764 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40664cb6-cf92-4184-a8b7-873b76ad5aa4 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:39.091151 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8107668-7349-461f-ae00-b317fad03fcc None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:39.129520 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af9a05f-b9f9-48fa-9fbe-d6e5cd1b2eb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:39.130190 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af9a05f-b9f9-48fa-9fbe-d6e5cd1b2eb7 None admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:39.130743 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af9a05f-b9f9-48fa-9fbe-d6e5cd1b2eb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:39.131023 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af9a05f-b9f9-48fa-9fbe-d6e5cd1b2eb7 None admin] PATH_INFO: `/v3/groups` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:39.131723 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af9a05f-b9f9-48fa-9fbe-d6e5cd1b2eb7 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:54:39.133336 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af9a05f-b9f9-48fa-9fbe-d6e5cd1b2eb7 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:54:39.154190 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 153/303] 10.222.0.46 () {60 vars in 1134 bytes} [Thu May 26 14:54:38 2022] POST /identity/v3/groups => generated 229 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 14:54:39.171410 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:39.236560 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40664cb6-cf92-4184-a8b7-873b76ad5aa4 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:39.243772 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00d1195-1ac7-4d00-b263-6bd9217188ab None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:39.244123 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00d1195-1ac7-4d00-b263-6bd9217188ab None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:39.244413 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00d1195-1ac7-4d00-b263-6bd9217188ab None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:39.254022 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 154/304] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 14:54:39 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 14:54:39.261686 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329e9c2b-76cf-4061-a343-0732aef577b8 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:39.262388 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329e9c2b-76cf-4061-a343-0732aef577b8 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:39.262388 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329e9c2b-76cf-4061-a343-0732aef577b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:39.269150 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:39.283585 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-329e9c2b-76cf-4061-a343-0732aef577b8 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:54:39.283809 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b66207d-ce73-4557-8efb-70df5039c86b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:39.284437 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b66207d-ce73-4557-8efb-70df5039c86b None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:39.284724 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b66207d-ce73-4557-8efb-70df5039c86b None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:39.285006 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b66207d-ce73-4557-8efb-70df5039c86b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:39.285286 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-329e9c2b-76cf-4061-a343-0732aef577b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:54:39.285895 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b66207d-ce73-4557-8efb-70df5039c86b None admin] RBAC: Adding query filter params (role.id=adb2011fe8f74ecbbb94465253e76c1c, user.id=454431a7cdaf4374a1090691a2b55560, scope.system=all) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:54:39.286216 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b66207d-ce73-4557-8efb-70df5039c86b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:54:39.288139 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b66207d-ce73-4557-8efb-70df5039c86b None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:54:39.303304 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 151/305] 10.222.0.46 () {56 vars in 1308 bytes} [Thu May 26 14:54:39 2022] GET /identity/v3/role_assignments?user.id=454431a7cdaf4374a1090691a2b55560&scope.system=all&role.id=adb2011fe8f74ecbbb94465253e76c1c => generated 226 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 14:54:39.371347 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 155/306] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 14:54:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 14:54:39.406458 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b66207d-ce73-4557-8efb-70df5039c86b None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:39.534614 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b66207d-ce73-4557-8efb-70df5039c86b None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:39.556906 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:39.576937 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70b0a5d-5641-4e1a-941b-1cbfc1d91da0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:39.578286 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70b0a5d-5641-4e1a-941b-1cbfc1d91da0 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:39.578896 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70b0a5d-5641-4e1a-941b-1cbfc1d91da0 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:39.579370 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70b0a5d-5641-4e1a-941b-1cbfc1d91da0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:39.591626 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70b0a5d-5641-4e1a-941b-1cbfc1d91da0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 14:54:39.599936 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-a70b0a5d-5641-4e1a-941b-1cbfc1d91da0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 26 14:54:39.599936 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:54:39.599936 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:54:39.599936 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:54:39.599936 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:54:39.599936 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:54:39.599936 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:54:39.599936 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:54:39.599936 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:54:39.599936 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:54:39.599936 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:54:39.599936 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:54:39.599936 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 14:54:39.599936 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 14:54:39.599936 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 14:54:39.599936 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 14:54:39.600839 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 14:54:39.600839 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 14:54:39.600839 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 14:54:39.600839 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 14:54:39.600839 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:54:39.600839 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:54:39.600839 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:54:39.600839 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:54:39.600839 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:54:39.600839 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:54:39.600839 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:54:39.600839 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 26 14:54:39.600839 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:54:39.600839 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 26 14:54:39.600839 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:54:39.600839 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:54:39.601650 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:54:39.601650 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 26 14:54:39.601650 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:54:39.601650 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 26 14:54:39.601650 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 14:54:39.601650 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 14:54:39.601650 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 14:54:39.601650 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 14:54:39.601650 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 14:54:39.601650 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 14:54:39.601650 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 26 14:54:39.601650 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 26 14:54:39.604387 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 152/307] 10.222.0.46 () {56 vars in 1099 bytes} [Thu May 26 14:54:39 2022] GET /identity/v3/roles/member => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:54:39.611042 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329e9c2b-76cf-4061-a343-0732aef577b8 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:39.752299 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329e9c2b-76cf-4061-a343-0732aef577b8 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:39.793425 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cea562-d80a-472a-af05-3e863827b555 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:39.794689 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cea562-d80a-472a-af05-3e863827b555 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:39.795186 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cea562-d80a-472a-af05-3e863827b555 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:39.795671 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cea562-d80a-472a-af05-3e863827b555 None admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:39.796636 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09cea562-d80a-472a-af05-3e863827b555 None admin] RBAC: Adding query filter params (name=member) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:54:39.797147 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09cea562-d80a-472a-af05-3e863827b555 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:54:39.798787 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09cea562-d80a-472a-af05-3e863827b555 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:54:39.800653 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:39.810321 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 156/308] 10.222.0.46 () {56 vars in 1101 bytes} [Thu May 26 14:54:39 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:54:39.816840 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70b0a5d-5641-4e1a-941b-1cbfc1d91da0 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:39.925534 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70b0a5d-5641-4e1a-941b-1cbfc1d91da0 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:39.962370 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ff91c7-88ee-4d92-9810-e9d2666dbdf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:39.963622 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ff91c7-88ee-4d92-9810-e9d2666dbdf3 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:39.964279 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ff91c7-88ee-4d92-9810-e9d2666dbdf3 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:39.965255 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ff91c7-88ee-4d92-9810-e9d2666dbdf3 None admin] PATH_INFO: `/v3/users/c3026d3cb3004d7abf348314511477bb` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:39.966697 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ff91c7-88ee-4d92-9810-e9d2666dbdf3 None admin] RBAC: Authorizing `identity:get_user(user_id=c3026d3cb3004d7abf348314511477bb)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:54:39.968351 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ff91c7-88ee-4d92-9810-e9d2666dbdf3 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:54:39.970626 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 153/309] 10.222.0.46 () {56 vars in 1177 bytes} [Thu May 26 14:54:39 2022] GET /identity/v3/users/c3026d3cb3004d7abf348314511477bb => generated 293 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:54:39.976439 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cea562-d80a-472a-af05-3e863827b555 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:40.094319 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cea562-d80a-472a-af05-3e863827b555 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:40.135527 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07085d2-7bda-46f0-a3cd-c4be8b9737c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:40.137211 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07085d2-7bda-46f0-a3cd-c4be8b9737c6 None admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:40.137714 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07085d2-7bda-46f0-a3cd-c4be8b9737c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:40.138171 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07085d2-7bda-46f0-a3cd-c4be8b9737c6 None admin] PATH_INFO: `/v3/system/users/c3026d3cb3004d7abf348314511477bb/roles/d4088d1845a24fbd9bd8975ce0b1d3d0` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:40.139842 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07085d2-7bda-46f0-a3cd-c4be8b9737c6 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=c3026d3cb3004d7abf348314511477bb, role_id=d4088d1845a24fbd9bd8975ce0b1d3d0)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:54:40.141395 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07085d2-7bda-46f0-a3cd-c4be8b9737c6 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:54:40.153193 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 157/310] 10.222.0.46 () {58 vars in 1334 bytes} [Thu May 26 14:54:39 2022] PUT /identity/v3/system/users/c3026d3cb3004d7abf348314511477bb/roles/d4088d1845a24fbd9bd8975ce0b1d3d0 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 14:54:40.240619 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7c8be7-f205-4c5c-a94b-2c3d6a9fb7a7 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:40.240915 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7c8be7-f205-4c5c-a94b-2c3d6a9fb7a7 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:40.241199 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7c8be7-f205-4c5c-a94b-2c3d6a9fb7a7 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:40.243048 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 154/311] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 14:54:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 14:54:40.249257 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889ab372-14f3-428c-b75d-69db2096f39c None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:40.249662 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889ab372-14f3-428c-b75d-69db2096f39c None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:40.249954 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889ab372-14f3-428c-b75d-69db2096f39c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:40.269582 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-889ab372-14f3-428c-b75d-69db2096f39c None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:54:40.270825 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-889ab372-14f3-428c-b75d-69db2096f39c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:54:40.359285 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:40.372232 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 158/312] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 14:54:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 14:54:40.396338 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7c8be7-f205-4c5c-a94b-2c3d6a9fb7a7 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:40.450511 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:40.508701 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7c8be7-f205-4c5c-a94b-2c3d6a9fb7a7 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:40.548325 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a4db11-a859-45d3-948c-be00df96b74d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:40.549130 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a4db11-a859-45d3-948c-be00df96b74d None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:40.549414 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a4db11-a859-45d3-948c-be00df96b74d None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:40.549720 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a4db11-a859-45d3-948c-be00df96b74d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:40.561389 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a4db11-a859-45d3-948c-be00df96b74d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 14:54:40.569718 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-d5a4db11-a859-45d3-948c-be00df96b74d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 26 14:54:40.569718 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:54:40.569718 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:54:40.569718 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:54:40.569718 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:54:40.569718 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:54:40.569718 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:54:40.569718 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:54:40.569718 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:54:40.569718 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:54:40.569718 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:54:40.569718 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:54:40.569718 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 14:54:40.569718 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 14:54:40.569718 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 14:54:40.569718 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 14:54:40.571862 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 14:54:40.571862 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 14:54:40.571862 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 14:54:40.571862 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 14:54:40.571862 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:54:40.571862 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:54:40.571862 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:54:40.571862 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:54:40.571862 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:54:40.571862 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:54:40.571862 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:54:40.571862 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 26 14:54:40.571862 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:54:40.571862 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 26 14:54:40.571862 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:54:40.571862 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:54:40.574394 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:54:40.574394 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 26 14:54:40.574394 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:54:40.574394 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 26 14:54:40.574394 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 14:54:40.574394 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 14:54:40.574394 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 14:54:40.574394 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 14:54:40.574394 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 14:54:40.574394 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 14:54:40.574394 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 26 14:54:40.574394 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 26 14:54:40.581491 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 155/313] 10.222.0.46 () {56 vars in 1099 bytes} [Thu May 26 14:54:40 2022] GET /identity/v3/roles/reader => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:54:40.581683 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889ab372-14f3-428c-b75d-69db2096f39c None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:40.716134 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889ab372-14f3-428c-b75d-69db2096f39c None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:40.761878 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2164ea5e-0c35-44a7-ae94-8132d4249bad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:40.762416 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2164ea5e-0c35-44a7-ae94-8132d4249bad None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:40.762673 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2164ea5e-0c35-44a7-ae94-8132d4249bad None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:40.762964 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2164ea5e-0c35-44a7-ae94-8132d4249bad None admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:40.763833 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2164ea5e-0c35-44a7-ae94-8132d4249bad None admin] RBAC: Adding query filter params (name=reader) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:54:40.764143 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2164ea5e-0c35-44a7-ae94-8132d4249bad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:54:40.765596 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2164ea5e-0c35-44a7-ae94-8132d4249bad None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:54:40.775703 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 159/314] 10.222.0.46 () {56 vars in 1101 bytes} [Thu May 26 14:54:40 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:54:40.781807 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a4db11-a859-45d3-948c-be00df96b74d None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:40.894181 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a4db11-a859-45d3-948c-be00df96b74d None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:40.905792 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:40.930859 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6690a7c7-0a87-41e6-870a-7e7a7cb6098b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:40.931743 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6690a7c7-0a87-41e6-870a-7e7a7cb6098b None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:40.932023 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6690a7c7-0a87-41e6-870a-7e7a7cb6098b None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:40.932296 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6690a7c7-0a87-41e6-870a-7e7a7cb6098b None admin] PATH_INFO: `/v3/users/b255355211fa4d81be18915a3ff1bef7` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:40.933565 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6690a7c7-0a87-41e6-870a-7e7a7cb6098b None admin] RBAC: Authorizing `identity:get_user(user_id=b255355211fa4d81be18915a3ff1bef7)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:54:40.935019 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6690a7c7-0a87-41e6-870a-7e7a7cb6098b None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:54:40.936880 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 156/315] 10.222.0.46 () {56 vars in 1177 bytes} [Thu May 26 14:54:40 2022] GET /identity/v3/users/b255355211fa4d81be18915a3ff1bef7 => generated 297 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:54:40.942353 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2164ea5e-0c35-44a7-ae94-8132d4249bad None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:41.001541 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:41.061256 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2164ea5e-0c35-44a7-ae94-8132d4249bad None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:41.100960 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5826245-b7b9-45c2-b443-a05ea043e930 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:41.101665 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5826245-b7b9-45c2-b443-a05ea043e930 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:41.101950 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5826245-b7b9-45c2-b443-a05ea043e930 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:41.102230 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5826245-b7b9-45c2-b443-a05ea043e930 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:41.189477 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5826245-b7b9-45c2-b443-a05ea043e930 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:54:41.191584 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5826245-b7b9-45c2-b443-a05ea043e930 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:54:41.245472 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:41.297346 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 160/316] 10.222.0.46 () {58 vars in 1282 bytes} [Thu May 26 14:54:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 14:54:41.302863 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6690a7c7-0a87-41e6-870a-7e7a7cb6098b None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:41.411692 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6690a7c7-0a87-41e6-870a-7e7a7cb6098b None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:41.451023 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ab6637-236e-4d1a-aa91-c79fc66d488e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:41.451837 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ab6637-236e-4d1a-aa91-c79fc66d488e None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:41.452116 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ab6637-236e-4d1a-aa91-c79fc66d488e None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:41.452389 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ab6637-236e-4d1a-aa91-c79fc66d488e None admin] PATH_INFO: `/v3/projects/94ad8a7442da4d658f58d1543ae06493` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:41.453739 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ab6637-236e-4d1a-aa91-c79fc66d488e None admin] RBAC: Authorizing `identity:get_project(project_id=94ad8a7442da4d658f58d1543ae06493)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:54:41.455231 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ab6637-236e-4d1a-aa91-c79fc66d488e None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:54:41.457057 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 157/317] 10.222.0.46 () {56 vars in 1186 bytes} [Thu May 26 14:54:41 2022] GET /identity/v3/projects/94ad8a7442da4d658f58d1543ae06493 => generated 302 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:54:41.463312 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5826245-b7b9-45c2-b443-a05ea043e930 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:41.584259 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5826245-b7b9-45c2-b443-a05ea043e930 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:41.634384 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d38702-2641-420c-8c91-f84dcf10b16a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:41.635049 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d38702-2641-420c-8c91-f84dcf10b16a None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:41.635157 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e32f784-36f1-4c3d-9661-c9841d37d01d None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:41.635333 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d38702-2641-420c-8c91-f84dcf10b16a None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:41.635427 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e32f784-36f1-4c3d-9661-c9841d37d01d None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:41.635610 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d38702-2641-420c-8c91-f84dcf10b16a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:41.635712 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e32f784-36f1-4c3d-9661-c9841d37d01d None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:41.638135 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 158/318] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 14:54:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 14:54:41.640968 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d38702-2641-420c-8c91-f84dcf10b16a None admin] RBAC: Adding query filter params (role.id=adb2011fe8f74ecbbb94465253e76c1c, scope.project.id=94ad8a7442da4d658f58d1543ae06493, user.id=b255355211fa4d81be18915a3ff1bef7) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:54:41.641328 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d38702-2641-420c-8c91-f84dcf10b16a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:54:41.643822 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d38702-2641-420c-8c91-f84dcf10b16a None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:54:41.645923 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c66d76-dc50-4b6d-8b42-f2622a30e2e2 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:41.646451 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c66d76-dc50-4b6d-8b42-f2622a30e2e2 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:41.646941 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c66d76-dc50-4b6d-8b42-f2622a30e2e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:41.652269 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 161/319] 10.222.0.46 () {56 vars in 1374 bytes} [Thu May 26 14:54:41 2022] GET /identity/v3/role_assignments?user.id=b255355211fa4d81be18915a3ff1bef7&scope.project.id=94ad8a7442da4d658f58d1543ae06493&role.id=adb2011fe8f74ecbbb94465253e76c1c => generated 608 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:54:41.679755 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-82c66d76-dc50-4b6d-8b42-f2622a30e2e2 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:54:41.681074 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-82c66d76-dc50-4b6d-8b42-f2622a30e2e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:54:41.690921 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f248cc-7d99-4cd0-af32-1b79dd7b94bd None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:41.691288 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f248cc-7d99-4cd0-af32-1b79dd7b94bd None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:41.691552 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f248cc-7d99-4cd0-af32-1b79dd7b94bd None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:41.693863 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 162/320] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 14:54:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 14:54:41.702466 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ef4bff-0a10-4c4f-9dc2-a90226b5e647 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:41.702714 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ef4bff-0a10-4c4f-9dc2-a90226b5e647 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:41.703392 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ef4bff-0a10-4c4f-9dc2-a90226b5e647 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:41.726254 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-e6ef4bff-0a10-4c4f-9dc2-a90226b5e647 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:54:41.727627 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-e6ef4bff-0a10-4c4f-9dc2-a90226b5e647 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:54:41.736657 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:41.753892 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:41.792853 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 159/321] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 14:54:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 14:54:41.814901 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c66d76-dc50-4b6d-8b42-f2622a30e2e2 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:41.829004 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 163/322] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 14:54:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 14:54:41.852360 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ef4bff-0a10-4c4f-9dc2-a90226b5e647 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:41.958673 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c66d76-dc50-4b6d-8b42-f2622a30e2e2 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:41.998153 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ef4bff-0a10-4c4f-9dc2-a90226b5e647 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:42.006653 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c110e2c4-3d5b-49c9-af55-53df0e074498 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:42.007476 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c110e2c4-3d5b-49c9-af55-53df0e074498 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:42.007758 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c110e2c4-3d5b-49c9-af55-53df0e074498 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:42.008031 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c110e2c4-3d5b-49c9-af55-53df0e074498 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:42.021309 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c110e2c4-3d5b-49c9-af55-53df0e074498 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 14:54:42.032633 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-c110e2c4-3d5b-49c9-af55-53df0e074498 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 26 14:54:42.032633 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:54:42.032633 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:54:42.032633 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:54:42.032633 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:54:42.032633 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:54:42.032633 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:54:42.032633 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:54:42.032633 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:54:42.032633 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:54:42.032633 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:54:42.032633 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:54:42.032633 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 14:54:42.032633 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 14:54:42.032633 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 14:54:42.032633 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 14:54:42.033902 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 14:54:42.033902 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 14:54:42.033902 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 14:54:42.033902 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 14:54:42.033902 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:54:42.033902 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:54:42.033902 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:54:42.033902 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:54:42.033902 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:54:42.033902 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:54:42.033902 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:54:42.033902 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 26 14:54:42.033902 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:54:42.033902 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 26 14:54:42.033902 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:54:42.033902 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:54:42.034698 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:54:42.034698 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 26 14:54:42.034698 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:54:42.034698 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 26 14:54:42.034698 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 14:54:42.034698 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 14:54:42.034698 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 14:54:42.034698 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 14:54:42.034698 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 14:54:42.034698 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 14:54:42.034698 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 26 14:54:42.034698 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 26 14:54:42.034698 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 160/323] 10.222.0.46 () {56 vars in 1099 bytes} [Thu May 26 14:54:41 2022] GET /identity/v3/roles/reader => generated 84 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:54:42.042296 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c110e2c4-3d5b-49c9-af55-53df0e074498 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:42.047398 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1c3c7e-4d6c-4e59-ab69-712d37da795b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:42.048213 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1c3c7e-4d6c-4e59-ab69-712d37da795b None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:42.048490 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1c3c7e-4d6c-4e59-ab69-712d37da795b None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:42.048759 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1c3c7e-4d6c-4e59-ab69-712d37da795b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:42.049989 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1c3c7e-4d6c-4e59-ab69-712d37da795b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:54:42.051454 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1c3c7e-4d6c-4e59-ab69-712d37da795b None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:54:42.053232 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 164/324] 10.222.0.46 () {56 vars in 1108 bytes} [Thu May 26 14:54:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 14:54:42.059003 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1c3c7e-4d6c-4e59-ab69-712d37da795b None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:42.077936 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:42.082978 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:42.158940 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c110e2c4-3d5b-49c9-af55-53df0e074498 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:42.187953 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1c3c7e-4d6c-4e59-ab69-712d37da795b None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:42.201081 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f62561-27c6-48f6-87bc-87782052d8cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:42.201886 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f62561-27c6-48f6-87bc-87782052d8cc None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:42.202179 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f62561-27c6-48f6-87bc-87782052d8cc None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:42.202463 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f62561-27c6-48f6-87bc-87782052d8cc None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:42.203227 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f62561-27c6-48f6-87bc-87782052d8cc None admin] RBAC: Adding query filter params (name=reader) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:54:42.203545 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f62561-27c6-48f6-87bc-87782052d8cc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:54:42.204937 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f62561-27c6-48f6-87bc-87782052d8cc None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:54:42.215505 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 161/325] 10.222.0.46 () {56 vars in 1101 bytes} [Thu May 26 14:54:42 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 14:54:42.220853 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f62561-27c6-48f6-87bc-87782052d8cc None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:42.227238 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e71c36-b42e-47a3-b6c7-ef9ff2814aa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:42.227966 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e71c36-b42e-47a3-b6c7-ef9ff2814aa7 None admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:42.228255 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e71c36-b42e-47a3-b6c7-ef9ff2814aa7 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:42.228530 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e71c36-b42e-47a3-b6c7-ef9ff2814aa7 None admin] PATH_INFO: `/v3/groups` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:42.229216 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e71c36-b42e-47a3-b6c7-ef9ff2814aa7 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:54:42.230735 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e71c36-b42e-47a3-b6c7-ef9ff2814aa7 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:54:42.248553 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 165/326] 10.222.0.46 () {60 vars in 1134 bytes} [Thu May 26 14:54:42 2022] POST /identity/v3/groups => generated 226 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 14:54:42.345894 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:42.354122 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f62561-27c6-48f6-87bc-87782052d8cc None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:42.393313 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebef8a4e-f155-406e-9db3-03c5214d4a7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:42.394247 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebef8a4e-f155-406e-9db3-03c5214d4a7d None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:42.394531 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebef8a4e-f155-406e-9db3-03c5214d4a7d None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:42.394807 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebef8a4e-f155-406e-9db3-03c5214d4a7d None admin] PATH_INFO: `/v3/users/454431a7cdaf4374a1090691a2b55560` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:42.396089 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebef8a4e-f155-406e-9db3-03c5214d4a7d None admin] RBAC: Authorizing `identity:get_user(user_id=454431a7cdaf4374a1090691a2b55560)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:54:42.397647 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebef8a4e-f155-406e-9db3-03c5214d4a7d None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:54:42.399561 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 162/327] 10.222.0.46 () {56 vars in 1177 bytes} [Thu May 26 14:54:42 2022] GET /identity/v3/users/454431a7cdaf4374a1090691a2b55560 => generated 293 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 14:54:42.405035 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e71c36-b42e-47a3-b6c7-ef9ff2814aa7 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:42.492355 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:42.526139 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e71c36-b42e-47a3-b6c7-ef9ff2814aa7 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:42.569680 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a91db0af-ffa0-47f9-b731-bd8934763671 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:42.571003 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91db0af-ffa0-47f9-b731-bd8934763671 None admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:42.571557 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91db0af-ffa0-47f9-b731-bd8934763671 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:42.572014 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a91db0af-ffa0-47f9-b731-bd8934763671 None admin] PATH_INFO: `/v3/system/users/454431a7cdaf4374a1090691a2b55560/roles/adb2011fe8f74ecbbb94465253e76c1c` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:42.574536 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91db0af-ffa0-47f9-b731-bd8934763671 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=454431a7cdaf4374a1090691a2b55560, role_id=adb2011fe8f74ecbbb94465253e76c1c)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:54:42.575973 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a91db0af-ffa0-47f9-b731-bd8934763671 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:54:42.587302 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 166/328] 10.222.0.46 () {58 vars in 1334 bytes} [Thu May 26 14:54:42 2022] PUT /identity/v3/system/users/454431a7cdaf4374a1090691a2b55560/roles/adb2011fe8f74ecbbb94465253e76c1c => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 14:54:42.770580 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eecea74-de43-4905-9332-a1bfc518dba5 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:42.771068 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eecea74-de43-4905-9332-a1bfc518dba5 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:42.771068 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eecea74-de43-4905-9332-a1bfc518dba5 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:42.772700 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 163/329] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 14:54:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 14:54:42.778460 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853afdb0-03f8-45b4-baf5-fae69c3393de None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:42.779340 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853afdb0-03f8-45b4-baf5-fae69c3393de None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:42.779824 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853afdb0-03f8-45b4-baf5-fae69c3393de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:42.799241 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-853afdb0-03f8-45b4-baf5-fae69c3393de None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:54:42.800440 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-853afdb0-03f8-45b4-baf5-fae69c3393de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:54:42.877943 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 167/330] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 14:54:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 14:54:42.901715 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eecea74-de43-4905-9332-a1bfc518dba5 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:43.045068 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eecea74-de43-4905-9332-a1bfc518dba5 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:43.097587 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3487f696-bb8f-4e4f-96cc-fc140f716b47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:43.098397 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3487f696-bb8f-4e4f-96cc-fc140f716b47 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:43.098681 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3487f696-bb8f-4e4f-96cc-fc140f716b47 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:43.098958 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3487f696-bb8f-4e4f-96cc-fc140f716b47 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:43.114653 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3487f696-bb8f-4e4f-96cc-fc140f716b47 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 14:54:43.123117 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-3487f696-bb8f-4e4f-96cc-fc140f716b47 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 26 14:54:43.123117 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:54:43.123117 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:54:43.123117 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:54:43.123117 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:54:43.123117 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:54:43.123117 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:54:43.123117 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:54:43.123117 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:54:43.123117 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:54:43.123117 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:54:43.123117 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:54:43.123117 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 14:54:43.123117 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 14:54:43.123117 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 14:54:43.123117 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 14:54:43.124038 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 14:54:43.124038 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 14:54:43.124038 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 14:54:43.124038 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 14:54:43.124038 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:54:43.124038 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:54:43.124038 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:54:43.124038 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:54:43.124038 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:54:43.124038 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:54:43.124038 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:54:43.124038 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 26 14:54:43.124038 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:54:43.124038 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 26 14:54:43.124038 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:54:43.124038 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:54:43.124818 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:54:43.124818 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 26 14:54:43.124818 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:54:43.124818 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 26 14:54:43.124818 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 14:54:43.124818 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 14:54:43.124818 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 14:54:43.124818 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 14:54:43.124818 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 14:54:43.124818 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 14:54:43.124818 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 26 14:54:43.124818 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 26 14:54:43.124818 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 164/331] 10.222.0.46 () {56 vars in 1099 bytes} [Thu May 26 14:54:42 2022] GET /identity/v3/roles/member => generated 84 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:54:43.132136 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853afdb0-03f8-45b4-baf5-fae69c3393de None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:43.230989 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:43.282639 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853afdb0-03f8-45b4-baf5-fae69c3393de None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:43.323217 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d7b3c8-06ce-4df3-b733-8a7c8cbf3448 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:43.323839 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d7b3c8-06ce-4df3-b733-8a7c8cbf3448 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:43.324118 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d7b3c8-06ce-4df3-b733-8a7c8cbf3448 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:43.324390 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d7b3c8-06ce-4df3-b733-8a7c8cbf3448 None admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:43.325138 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d7b3c8-06ce-4df3-b733-8a7c8cbf3448 None admin] RBAC: Adding query filter params (name=member) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:54:43.325439 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d7b3c8-06ce-4df3-b733-8a7c8cbf3448 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:54:43.326909 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d7b3c8-06ce-4df3-b733-8a7c8cbf3448 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:54:43.344315 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 168/332] 10.222.0.46 () {56 vars in 1101 bytes} [Thu May 26 14:54:43 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:54:43.349491 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3487f696-bb8f-4e4f-96cc-fc140f716b47 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:43.409665 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:43.540373 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3487f696-bb8f-4e4f-96cc-fc140f716b47 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:43.598618 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c09f4bb-70cc-47fc-97cf-5339ce64cafb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:43.599828 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c09f4bb-70cc-47fc-97cf-5339ce64cafb None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:43.600303 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c09f4bb-70cc-47fc-97cf-5339ce64cafb None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:43.600770 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c09f4bb-70cc-47fc-97cf-5339ce64cafb None admin] PATH_INFO: `/v3/users/c3026d3cb3004d7abf348314511477bb` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:43.603329 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c09f4bb-70cc-47fc-97cf-5339ce64cafb None admin] RBAC: Authorizing `identity:get_user(user_id=c3026d3cb3004d7abf348314511477bb)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:54:43.605111 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c09f4bb-70cc-47fc-97cf-5339ce64cafb None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:54:43.607306 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 165/333] 10.222.0.46 () {56 vars in 1177 bytes} [Thu May 26 14:54:43 2022] GET /identity/v3/users/c3026d3cb3004d7abf348314511477bb => generated 293 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:54:43.613252 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d7b3c8-06ce-4df3-b733-8a7c8cbf3448 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:43.779863 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d7b3c8-06ce-4df3-b733-8a7c8cbf3448 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:43.814530 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:43.825199 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af28dd3-f982-48db-8a80-eb475b6957a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:43.825895 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af28dd3-f982-48db-8a80-eb475b6957a4 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:43.826179 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af28dd3-f982-48db-8a80-eb475b6957a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:43.826454 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af28dd3-f982-48db-8a80-eb475b6957a4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:43.827310 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af28dd3-f982-48db-8a80-eb475b6957a4 None admin] RBAC: Adding query filter params (role.id=d4088d1845a24fbd9bd8975ce0b1d3d0, user.id=c3026d3cb3004d7abf348314511477bb, scope.system=all) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:54:43.827604 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af28dd3-f982-48db-8a80-eb475b6957a4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:54:43.829041 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af28dd3-f982-48db-8a80-eb475b6957a4 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:54:43.842972 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 169/334] 10.222.0.46 () {56 vars in 1308 bytes} [Thu May 26 14:54:43 2022] GET /identity/v3/role_assignments?user.id=c3026d3cb3004d7abf348314511477bb&scope.system=all&role.id=d4088d1845a24fbd9bd8975ce0b1d3d0 => generated 510 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:54:45.434984 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5312509-141d-415c-b700-bfe52306a7d4 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:45.435557 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5312509-141d-415c-b700-bfe52306a7d4 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:45.435557 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5312509-141d-415c-b700-bfe52306a7d4 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:45.437654 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 166/335] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 14:54:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 14:54:45.443250 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a0782a-402d-466b-85df-3121454f1416 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:45.443753 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a0782a-402d-466b-85df-3121454f1416 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:45.444013 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a0782a-402d-466b-85df-3121454f1416 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:45.468314 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-a8a0782a-402d-466b-85df-3121454f1416 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:54:45.469620 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-a8a0782a-402d-466b-85df-3121454f1416 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:54:45.573791 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 170/336] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 14:54:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 14:54:45.619415 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5312509-141d-415c-b700-bfe52306a7d4 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:45.732025 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:45.746188 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5312509-141d-415c-b700-bfe52306a7d4 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:45.788437 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa225869-0b04-4c7a-8e33-140b86ee5160 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:45.789254 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa225869-0b04-4c7a-8e33-140b86ee5160 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:45.789565 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa225869-0b04-4c7a-8e33-140b86ee5160 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:45.789846 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa225869-0b04-4c7a-8e33-140b86ee5160 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:45.805769 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7c71cf-fe4c-49bb-9572-1aa2408f4a24 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:45.806383 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7c71cf-fe4c-49bb-9572-1aa2408f4a24 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:45.806781 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa225869-0b04-4c7a-8e33-140b86ee5160 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 14:54:45.807088 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7c71cf-fe4c-49bb-9572-1aa2408f4a24 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:45.808969 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 171/337] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 14:54:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 14:54:45.813669 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4fc453-b2a8-42d3-89a1-e5d700a2cf12 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:45.814196 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4fc453-b2a8-42d3-89a1-e5d700a2cf12 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:45.814729 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4fc453-b2a8-42d3-89a1-e5d700a2cf12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:45.820533 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-fa225869-0b04-4c7a-8e33-140b86ee5160 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 26 14:54:45.820533 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:54:45.820533 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:54:45.820533 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:54:45.820533 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:54:45.820533 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:54:45.820533 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:54:45.820533 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:54:45.820533 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:54:45.820533 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:54:45.820533 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:54:45.820533 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:54:45.820533 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 14:54:45.820533 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 14:54:45.820533 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 14:54:45.820533 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 14:54:45.822028 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 14:54:45.822028 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 14:54:45.822028 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 14:54:45.822028 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 14:54:45.822028 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:54:45.822028 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:54:45.822028 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:54:45.822028 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:54:45.822028 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:54:45.822028 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:54:45.822028 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:54:45.822028 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 26 14:54:45.822028 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:54:45.822028 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 26 14:54:45.822028 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:54:45.822028 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:54:45.822829 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:54:45.822829 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 26 14:54:45.822829 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:54:45.822829 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 26 14:54:45.822829 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 14:54:45.822829 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 14:54:45.822829 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 14:54:45.822829 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 14:54:45.822829 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 14:54:45.822829 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 14:54:45.822829 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 26 14:54:45.822829 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 26 14:54:45.822829 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 167/338] 10.222.0.46 () {56 vars in 1099 bytes} [Thu May 26 14:54:45 2022] GET /identity/v3/roles/member => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:54:45.832238 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa225869-0b04-4c7a-8e33-140b86ee5160 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:45.847548 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-db4fc453-b2a8-42d3-89a1-e5d700a2cf12 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:54:45.849179 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-db4fc453-b2a8-42d3-89a1-e5d700a2cf12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:54:45.972100 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa225869-0b04-4c7a-8e33-140b86ee5160 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:45.979232 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 172/339] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 14:54:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 14:54:46.018888 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db6706d-0b9d-40bb-b8d1-31ff20079147 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:46.019489 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db6706d-0b9d-40bb-b8d1-31ff20079147 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:46.019776 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db6706d-0b9d-40bb-b8d1-31ff20079147 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:46.020054 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db6706d-0b9d-40bb-b8d1-31ff20079147 None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:46.020810 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db6706d-0b9d-40bb-b8d1-31ff20079147 None admin] RBAC: Adding query filter params (name=member) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:54:46.021115 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db6706d-0b9d-40bb-b8d1-31ff20079147 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:54:46.022538 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db6706d-0b9d-40bb-b8d1-31ff20079147 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:54:46.031502 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 168/340] 10.222.0.46 () {56 vars in 1101 bytes} [Thu May 26 14:54:45 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 14:54:46.039529 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4fc453-b2a8-42d3-89a1-e5d700a2cf12 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:46.044625 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db6706d-0b9d-40bb-b8d1-31ff20079147 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:46.069963 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:46.145035 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:46.168853 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db6706d-0b9d-40bb-b8d1-31ff20079147 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:46.209129 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1be8de-7673-43e2-9dc1-43e2af2d5aba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:46.209536 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1be8de-7673-43e2-9dc1-43e2af2d5aba None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:46.210030 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1be8de-7673-43e2-9dc1-43e2af2d5aba None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:46.210506 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1be8de-7673-43e2-9dc1-43e2af2d5aba None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:46.222309 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4fc453-b2a8-42d3-89a1-e5d700a2cf12 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:46.223009 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1be8de-7673-43e2-9dc1-43e2af2d5aba None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 14:54:46.231531 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-fe1be8de-7673-43e2-9dc1-43e2af2d5aba None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 14:54:46.231531 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:54:46.231531 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:54:46.231531 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:54:46.231531 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:54:46.231531 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:54:46.231531 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:54:46.231531 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:54:46.231531 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:54:46.231531 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:54:46.231531 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:54:46.231531 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:54:46.231531 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 14:54:46.231531 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 14:54:46.231531 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 14:54:46.231531 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 14:54:46.232681 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 14:54:46.232681 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 14:54:46.232681 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 14:54:46.232681 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 14:54:46.232681 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:54:46.232681 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:54:46.232681 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:54:46.232681 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:54:46.232681 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:54:46.232681 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:54:46.232681 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:54:46.232681 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 26 14:54:46.232681 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:54:46.232681 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 26 14:54:46.232681 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:54:46.232681 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:54:46.233696 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:54:46.233696 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 26 14:54:46.233696 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:54:46.233696 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 26 14:54:46.233696 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 14:54:46.233696 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 14:54:46.233696 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 14:54:46.233696 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 14:54:46.233696 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 14:54:46.233696 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 14:54:46.233696 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 14:54:46.233696 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 26 14:54:46.233696 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 169/341] 10.222.0.46 () {56 vars in 1114 bytes} [Thu May 26 14:54:46 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 14:54:46.242630 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1be8de-7673-43e2-9dc1-43e2af2d5aba None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:46.277243 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f785d1a-c88f-47f6-92bd-e60d412ef1b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:46.278185 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f785d1a-c88f-47f6-92bd-e60d412ef1b2 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:46.278615 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f785d1a-c88f-47f6-92bd-e60d412ef1b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:46.279187 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f785d1a-c88f-47f6-92bd-e60d412ef1b2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:46.339570 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:46.377488 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1be8de-7673-43e2-9dc1-43e2af2d5aba None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:46.383626 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f785d1a-c88f-47f6-92bd-e60d412ef1b2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:54:46.385309 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f785d1a-c88f-47f6-92bd-e60d412ef1b2 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:54:46.458628 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:46.484873 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db7d95a-e5f2-4538-877f-05c3ce387a0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:46.487650 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db7d95a-e5f2-4538-877f-05c3ce387a0e None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:46.487650 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db7d95a-e5f2-4538-877f-05c3ce387a0e None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:46.487650 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db7d95a-e5f2-4538-877f-05c3ce387a0e None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:46.488298 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db7d95a-e5f2-4538-877f-05c3ce387a0e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:54:46.488802 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db7d95a-e5f2-4538-877f-05c3ce387a0e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:54:46.491717 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db7d95a-e5f2-4538-877f-05c3ce387a0e None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:54:46.521500 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 170/342] 10.222.0.46 () {56 vars in 1111 bytes} [Thu May 26 14:54:46 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:54:46.523091 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 173/343] 10.222.0.46 () {58 vars in 1282 bytes} [Thu May 26 14:54:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 484 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 14:54:46.527750 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db7d95a-e5f2-4538-877f-05c3ce387a0e None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:46.531447 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f785d1a-c88f-47f6-92bd-e60d412ef1b2 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:46.793833 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f785d1a-c88f-47f6-92bd-e60d412ef1b2 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:46.807366 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db7d95a-e5f2-4538-877f-05c3ce387a0e None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:46.871170 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0261f6-7373-4095-93b5-2d79317671b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:46.872080 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0261f6-7373-4095-93b5-2d79317671b1 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:46.872541 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0261f6-7373-4095-93b5-2d79317671b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:46.872966 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0261f6-7373-4095-93b5-2d79317671b1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:46.875117 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfc6cfa-2d0a-4539-b382-1c92928d99f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:46.876011 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfc6cfa-2d0a-4539-b382-1c92928d99f5 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:46.876324 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfc6cfa-2d0a-4539-b382-1c92928d99f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:46.876569 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfc6cfa-2d0a-4539-b382-1c92928d99f5 None admin] PATH_INFO: `/v3/projects/e474420e5c124a3f86383fce1ef4c4fb` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:46.878346 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:46.882496 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfc6cfa-2d0a-4539-b382-1c92928d99f5 None admin] RBAC: Authorizing `identity:get_project(project_id=e474420e5c124a3f86383fce1ef4c4fb)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:54:46.884157 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebfc6cfa-2d0a-4539-b382-1c92928d99f5 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:54:46.886292 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 174/344] 10.222.0.46 () {56 vars in 1186 bytes} [Thu May 26 14:54:46 2022] GET /identity/v3/projects/e474420e5c124a3f86383fce1ef4c4fb => generated 298 bytes in 355 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:54:46.887706 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60e5f79-0115-4813-92d3-8f0d2ce35986 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:46.888141 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60e5f79-0115-4813-92d3-8f0d2ce35986 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:46.888581 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60e5f79-0115-4813-92d3-8f0d2ce35986 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:46.890148 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 175/345] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 14:54:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 14:54:46.894505 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60e5f79-0115-4813-92d3-8f0d2ce35986 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:46.928980 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:46.956391 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0261f6-7373-4095-93b5-2d79317671b1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:54:46.957975 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0261f6-7373-4095-93b5-2d79317671b1 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:54:47.075180 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 171/346] 10.222.0.46 () {58 vars in 1282 bytes} [Thu May 26 14:54:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 548 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 14:54:47.076461 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8d5d82-8463-4096-88bb-2c4b88ec09e6 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:47.076918 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8d5d82-8463-4096-88bb-2c4b88ec09e6 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:47.077343 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8d5d82-8463-4096-88bb-2c4b88ec09e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:47.101667 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-3c8d5d82-8463-4096-88bb-2c4b88ec09e6 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:54:47.103299 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-3c8d5d82-8463-4096-88bb-2c4b88ec09e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:54:47.139009 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60e5f79-0115-4813-92d3-8f0d2ce35986 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:47.144904 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:47.178483 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f236b73f-e929-4b4e-8d03-24c5892d23fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:47.179361 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f236b73f-e929-4b4e-8d03-24c5892d23fb None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:47.179646 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f236b73f-e929-4b4e-8d03-24c5892d23fb None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:47.179927 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f236b73f-e929-4b4e-8d03-24c5892d23fb None admin] PATH_INFO: `/v3/groups/4c84bf7a4c4f41da83d400606d236080` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:47.187726 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f236b73f-e929-4b4e-8d03-24c5892d23fb None admin] RBAC: Authorizing `identity:get_group(group_id=4c84bf7a4c4f41da83d400606d236080)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:54:47.189575 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f236b73f-e929-4b4e-8d03-24c5892d23fb None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:54:47.191716 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 176/347] 10.222.0.46 () {56 vars in 1180 bytes} [Thu May 26 14:54:46 2022] GET /identity/v3/groups/4c84bf7a4c4f41da83d400606d236080 => generated 226 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 14:54:47.192906 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f236b73f-e929-4b4e-8d03-24c5892d23fb None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:47.221590 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 172/348] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 14:54:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 14:54:47.224170 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15204a1-40ce-4f3e-9bc5-6e9173680ee9 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:47.224784 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15204a1-40ce-4f3e-9bc5-6e9173680ee9 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:47.225360 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15204a1-40ce-4f3e-9bc5-6e9173680ee9 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:47.228398 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 173/349] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 14:54:47 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 14:54:47.229839 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15204a1-40ce-4f3e-9bc5-6e9173680ee9 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:47.353234 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:47.395114 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f236b73f-e929-4b4e-8d03-24c5892d23fb None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:47.397214 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15204a1-40ce-4f3e-9bc5-6e9173680ee9 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:47.443067 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a15dcb-1d88-41d9-ac0d-652f672f8a98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:47.444020 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a15dcb-1d88-41d9-ac0d-652f672f8a98 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:47.444466 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a15dcb-1d88-41d9-ac0d-652f672f8a98 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:47.444888 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a15dcb-1d88-41d9-ac0d-652f672f8a98 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:47.446492 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a15dcb-1d88-41d9-ac0d-652f672f8a98 None admin] RBAC: Adding query filter params (group.id=4c84bf7a4c4f41da83d400606d236080, role.id=d4088d1845a24fbd9bd8975ce0b1d3d0, scope.project.id=e474420e5c124a3f86383fce1ef4c4fb) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:54:47.446960 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a15dcb-1d88-41d9-ac0d-652f672f8a98 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:54:47.449128 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a15dcb-1d88-41d9-ac0d-652f672f8a98 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:54:47.457996 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 174/350] 10.222.0.46 () {56 vars in 1376 bytes} [Thu May 26 14:54:47 2022] GET /identity/v3/role_assignments?group.id=4c84bf7a4c4f41da83d400606d236080&scope.project.id=e474420e5c124a3f86383fce1ef4c4fb&role.id=d4088d1845a24fbd9bd8975ce0b1d3d0 => generated 260 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:54:47.459585 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce1e93b-8ede-4d8b-b83c-13d2e13ce7c8 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:47.460005 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce1e93b-8ede-4d8b-b83c-13d2e13ce7c8 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:47.460411 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce1e93b-8ede-4d8b-b83c-13d2e13ce7c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:47.470001 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1242cfa6-2054-4a18-a43c-1c3ead12406c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:47.470988 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1242cfa6-2054-4a18-a43c-1c3ead12406c None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:47.471305 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1242cfa6-2054-4a18-a43c-1c3ead12406c None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:47.471624 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1242cfa6-2054-4a18-a43c-1c3ead12406c None admin] PATH_INFO: `/v3/projects/e474420e5c124a3f86383fce1ef4c4fb` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:47.473670 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1242cfa6-2054-4a18-a43c-1c3ead12406c None admin] RBAC: Authorizing `identity:get_project(project_id=e474420e5c124a3f86383fce1ef4c4fb)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:54:47.475301 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1242cfa6-2054-4a18-a43c-1c3ead12406c None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:54:47.481121 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 177/351] 10.222.0.46 () {56 vars in 1186 bytes} [Thu May 26 14:54:47 2022] GET /identity/v3/projects/e474420e5c124a3f86383fce1ef4c4fb => generated 298 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:54:47.482302 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1242cfa6-2054-4a18-a43c-1c3ead12406c None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:47.488109 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-1ce1e93b-8ede-4d8b-b83c-13d2e13ce7c8 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:54:47.490014 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-1ce1e93b-8ede-4d8b-b83c-13d2e13ce7c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:54:47.534362 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:47.616637 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 175/352] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 14:54:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 14:54:47.617929 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce1e93b-8ede-4d8b-b83c-13d2e13ce7c8 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:47.640562 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1242cfa6-2054-4a18-a43c-1c3ead12406c None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:47.691765 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffac9e82-9d42-4d48-bc26-1d0f1a8cc426 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:47.692659 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffac9e82-9d42-4d48-bc26-1d0f1a8cc426 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:47.692953 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffac9e82-9d42-4d48-bc26-1d0f1a8cc426 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:47.693231 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffac9e82-9d42-4d48-bc26-1d0f1a8cc426 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:47.705961 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffac9e82-9d42-4d48-bc26-1d0f1a8cc426 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 14:54:47.714569 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-ffac9e82-9d42-4d48-bc26-1d0f1a8cc426 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 26 14:54:47.714569 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:54:47.714569 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:54:47.714569 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:54:47.714569 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:54:47.714569 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:54:47.714569 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:54:47.714569 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:54:47.714569 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:54:47.714569 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:54:47.714569 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:54:47.714569 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:54:47.714569 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 14:54:47.714569 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 14:54:47.714569 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 14:54:47.714569 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 14:54:47.715572 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 14:54:47.715572 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 14:54:47.715572 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 14:54:47.715572 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 14:54:47.715572 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:54:47.715572 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:54:47.715572 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:54:47.715572 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:54:47.715572 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:54:47.715572 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:54:47.715572 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:54:47.715572 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 26 14:54:47.715572 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:54:47.715572 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 26 14:54:47.715572 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:54:47.715572 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:54:47.716380 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:54:47.716380 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 26 14:54:47.716380 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:54:47.716380 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 26 14:54:47.716380 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 14:54:47.716380 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 14:54:47.716380 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 14:54:47.716380 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 14:54:47.716380 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 14:54:47.716380 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 14:54:47.716380 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 26 14:54:47.716380 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 26 14:54:47.716380 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 178/353] 10.222.0.46 () {56 vars in 1099 bytes} [Thu May 26 14:54:47 2022] GET /identity/v3/roles/reader => generated 84 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:54:47.721335 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3f9c76-007e-4db8-86f3-0d38599b8874 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:47.721706 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3f9c76-007e-4db8-86f3-0d38599b8874 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:47.721925 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3f9c76-007e-4db8-86f3-0d38599b8874 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:47.723647 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 179/354] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 14:54:47 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 14:54:47.728066 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3f9c76-007e-4db8-86f3-0d38599b8874 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:47.741747 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:47.775895 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce1e93b-8ede-4d8b-b83c-13d2e13ce7c8 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:47.837365 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7014cbd6-f4e9-46b5-8a31-6e549c96fcf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:47.838604 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7014cbd6-f4e9-46b5-8a31-6e549c96fcf8 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:47.839111 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7014cbd6-f4e9-46b5-8a31-6e549c96fcf8 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:47.839596 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7014cbd6-f4e9-46b5-8a31-6e549c96fcf8 None admin] PATH_INFO: `/v3/groups/4c84bf7a4c4f41da83d400606d236080` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:47.840993 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7014cbd6-f4e9-46b5-8a31-6e549c96fcf8 None admin] RBAC: Authorizing `identity:get_group(group_id=4c84bf7a4c4f41da83d400606d236080)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:54:47.844390 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7014cbd6-f4e9-46b5-8a31-6e549c96fcf8 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:54:47.846399 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 176/355] 10.222.0.46 () {56 vars in 1180 bytes} [Thu May 26 14:54:47 2022] GET /identity/v3/groups/4c84bf7a4c4f41da83d400606d236080 => generated 226 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:54:47.847410 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7014cbd6-f4e9-46b5-8a31-6e549c96fcf8 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:47.865318 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3f9c76-007e-4db8-86f3-0d38599b8874 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:47.882394 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:47.927076 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8b12e9-0205-424f-ac44-381a207f4c2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:47.928015 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8b12e9-0205-424f-ac44-381a207f4c2f None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:47.928304 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8b12e9-0205-424f-ac44-381a207f4c2f None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:47.928586 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8b12e9-0205-424f-ac44-381a207f4c2f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:47.946095 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8b12e9-0205-424f-ac44-381a207f4c2f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 14:54:47.960887 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-7a8b12e9-0205-424f-ac44-381a207f4c2f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 14:54:47.960887 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:54:47.960887 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:54:47.960887 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:54:47.960887 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:54:47.960887 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:54:47.960887 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:54:47.960887 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:54:47.960887 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:54:47.960887 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:54:47.960887 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:54:47.960887 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:54:47.960887 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 14:54:47.960887 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 14:54:47.960887 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 14:54:47.960887 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 14:54:47.961852 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 14:54:47.961852 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 14:54:47.961852 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 14:54:47.961852 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 14:54:47.961852 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:54:47.961852 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:54:47.961852 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:54:47.961852 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:54:47.961852 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:54:47.961852 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:54:47.961852 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:54:47.961852 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 26 14:54:47.961852 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:54:47.961852 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 26 14:54:47.961852 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:54:47.961852 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:54:47.962644 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:54:47.962644 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 26 14:54:47.962644 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:54:47.962644 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 26 14:54:47.962644 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 14:54:47.962644 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 14:54:47.962644 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 14:54:47.962644 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 14:54:47.962644 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 14:54:47.962644 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 14:54:47.962644 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 14:54:47.962644 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 26 14:54:47.966298 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 180/356] 10.222.0.46 () {56 vars in 1114 bytes} [Thu May 26 14:54:47 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:54:47.968200 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fed6626-d9fb-4308-a046-247a1a0d6718 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:47.968496 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fed6626-d9fb-4308-a046-247a1a0d6718 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:47.968779 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fed6626-d9fb-4308-a046-247a1a0d6718 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:47.992972 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-3fed6626-d9fb-4308-a046-247a1a0d6718 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:54:47.994202 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-3fed6626-d9fb-4308-a046-247a1a0d6718 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:54:48.000440 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7014cbd6-f4e9-46b5-8a31-6e549c96fcf8 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:48.049486 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7bd93e-65b2-4f10-a3e8-9ed653301f49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:48.050493 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7bd93e-65b2-4f10-a3e8-9ed653301f49 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:48.050869 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7bd93e-65b2-4f10-a3e8-9ed653301f49 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:48.051153 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7bd93e-65b2-4f10-a3e8-9ed653301f49 None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:48.052123 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7bd93e-65b2-4f10-a3e8-9ed653301f49 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:54:48.052491 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7bd93e-65b2-4f10-a3e8-9ed653301f49 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:54:48.054602 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b7bd93e-65b2-4f10-a3e8-9ed653301f49 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:54:48.066189 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 177/357] 10.222.0.46 () {56 vars in 1101 bytes} [Thu May 26 14:54:47 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:54:48.070825 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7bd93e-65b2-4f10-a3e8-9ed653301f49 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:48.079363 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:48.086077 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 181/358] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 14:54:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 14:54:48.087288 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fed6626-d9fb-4308-a046-247a1a0d6718 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:48.202629 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:48.215279 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7bd93e-65b2-4f10-a3e8-9ed653301f49 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:48.233932 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fed6626-d9fb-4308-a046-247a1a0d6718 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:48.266877 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12197280-e006-4093-a5f2-4a8d30266491 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:48.267150 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12197280-e006-4093-a5f2-4a8d30266491 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:48.267484 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12197280-e006-4093-a5f2-4a8d30266491 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:48.267763 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12197280-e006-4093-a5f2-4a8d30266491 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:48.268600 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12197280-e006-4093-a5f2-4a8d30266491 None admin] RBAC: Adding query filter params (group.id=4c84bf7a4c4f41da83d400606d236080, role.id=160115b2143c4b389d39cd93f43e1d01, scope.project.id=e474420e5c124a3f86383fce1ef4c4fb) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:54:48.268911 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12197280-e006-4093-a5f2-4a8d30266491 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:54:48.271432 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12197280-e006-4093-a5f2-4a8d30266491 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:54:48.278505 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 178/359] 10.222.0.46 () {56 vars in 1376 bytes} [Thu May 26 14:54:48 2022] GET /identity/v3/role_assignments?group.id=4c84bf7a4c4f41da83d400606d236080&scope.project.id=e474420e5c124a3f86383fce1ef4c4fb&role.id=160115b2143c4b389d39cd93f43e1d01 => generated 260 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:54:48.279816 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac755c3-8401-44a4-a83a-a8f529e1851c None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:48.280102 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac755c3-8401-44a4-a83a-a8f529e1851c None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:48.280410 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cac755c3-8401-44a4-a83a-a8f529e1851c None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:48.282028 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 179/360] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 14:54:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 14:54:48.282535 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac755c3-8401-44a4-a83a-a8f529e1851c None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:48.284353 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59b52a0-0dfc-4473-b408-c331b6546ad6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:48.285039 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59b52a0-0dfc-4473-b408-c331b6546ad6 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:48.285362 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59b52a0-0dfc-4473-b408-c331b6546ad6 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:48.285948 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59b52a0-0dfc-4473-b408-c331b6546ad6 None admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:48.287742 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59b52a0-0dfc-4473-b408-c331b6546ad6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:54:48.288080 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59b52a0-0dfc-4473-b408-c331b6546ad6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:54:48.289626 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59b52a0-0dfc-4473-b408-c331b6546ad6 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:54:48.302662 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 182/361] 10.222.0.46 () {56 vars in 1111 bytes} [Thu May 26 14:54:48 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:54:48.303439 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59b52a0-0dfc-4473-b408-c331b6546ad6 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:48.389666 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:48.437544 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cac755c3-8401-44a4-a83a-a8f529e1851c None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:48.446431 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59b52a0-0dfc-4473-b408-c331b6546ad6 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:48.484864 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64efc9eb-c1e4-4434-9a9d-63186735dda5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:48.486413 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64efc9eb-c1e4-4434-9a9d-63186735dda5 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:48.486849 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64efc9eb-c1e4-4434-9a9d-63186735dda5 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:48.487186 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27107c26-0b15-4c69-b76c-2c3f60eae2fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:48.487454 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64efc9eb-c1e4-4434-9a9d-63186735dda5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:48.488648 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27107c26-0b15-4c69-b76c-2c3f60eae2fb None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:48.488648 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27107c26-0b15-4c69-b76c-2c3f60eae2fb None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:48.489160 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27107c26-0b15-4c69-b76c-2c3f60eae2fb None admin] PATH_INFO: `/v3/users/454431a7cdaf4374a1090691a2b55560` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:48.490158 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27107c26-0b15-4c69-b76c-2c3f60eae2fb None admin] RBAC: Authorizing `identity:get_user(user_id=454431a7cdaf4374a1090691a2b55560)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:54:48.492507 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27107c26-0b15-4c69-b76c-2c3f60eae2fb None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:54:48.497542 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 180/362] 10.222.0.46 () {56 vars in 1177 bytes} [Thu May 26 14:54:48 2022] GET /identity/v3/users/454431a7cdaf4374a1090691a2b55560 => generated 293 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:54:48.499132 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c79eb5-a897-40a1-a2da-6a149929da10 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:48.499570 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c79eb5-a897-40a1-a2da-6a149929da10 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:48.500335 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c79eb5-a897-40a1-a2da-6a149929da10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:48.509048 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64efc9eb-c1e4-4434-9a9d-63186735dda5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 14:54:48.517922 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-64efc9eb-c1e4-4434-9a9d-63186735dda5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 26 14:54:48.517922 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:54:48.517922 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:54:48.517922 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:54:48.517922 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:54:48.517922 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:54:48.517922 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:54:48.517922 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:54:48.517922 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:54:48.517922 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:54:48.517922 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:54:48.517922 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:54:48.517922 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 14:54:48.517922 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 14:54:48.517922 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 14:54:48.517922 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 14:54:48.518886 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 14:54:48.518886 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 14:54:48.518886 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 14:54:48.518886 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 14:54:48.518886 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:54:48.518886 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:54:48.518886 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:54:48.518886 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:54:48.518886 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:54:48.518886 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:54:48.518886 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:54:48.518886 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 26 14:54:48.518886 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:54:48.518886 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 26 14:54:48.518886 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:54:48.518886 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:54:48.519695 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:54:48.519695 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 26 14:54:48.519695 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:54:48.519695 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 26 14:54:48.519695 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 14:54:48.519695 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 14:54:48.519695 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 14:54:48.519695 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 14:54:48.519695 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 14:54:48.519695 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 14:54:48.519695 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 26 14:54:48.519695 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 26 14:54:48.520830 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-c6c79eb5-a897-40a1-a2da-6a149929da10 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:54:48.521978 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-c6c79eb5-a897-40a1-a2da-6a149929da10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:54:48.522638 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 183/363] 10.222.0.46 () {56 vars in 1099 bytes} [Thu May 26 14:54:48 2022] GET /identity/v3/roles/member => generated 84 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:54:48.523858 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64efc9eb-c1e4-4434-9a9d-63186735dda5 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:48.541492 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:48.615092 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 181/364] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 14:54:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 14:54:48.616006 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c79eb5-a897-40a1-a2da-6a149929da10 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:48.658405 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64efc9eb-c1e4-4434-9a9d-63186735dda5 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:48.686440 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:48.705550 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e654826d-f709-4767-b553-5105ed77dbe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:48.706212 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e654826d-f709-4767-b553-5105ed77dbe0 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:48.706498 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e654826d-f709-4767-b553-5105ed77dbe0 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:48.706776 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e654826d-f709-4767-b553-5105ed77dbe0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:48.742921 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c79eb5-a897-40a1-a2da-6a149929da10 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:48.780897 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5312b1-38a7-4d67-8a94-71172559336c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:48.781980 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5312b1-38a7-4d67-8a94-71172559336c None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:48.782483 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5312b1-38a7-4d67-8a94-71172559336c None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:48.782958 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5312b1-38a7-4d67-8a94-71172559336c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:48.784013 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5312b1-38a7-4d67-8a94-71172559336c None admin] RBAC: Adding query filter params (role.id=adb2011fe8f74ecbbb94465253e76c1c, user.id=454431a7cdaf4374a1090691a2b55560, scope.system=all) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:54:48.784527 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5312b1-38a7-4d67-8a94-71172559336c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:54:48.786198 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5312b1-38a7-4d67-8a94-71172559336c None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:54:48.798171 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e654826d-f709-4767-b553-5105ed77dbe0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:54:48.799707 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e654826d-f709-4767-b553-5105ed77dbe0 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:54:48.800445 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 182/365] 10.222.0.46 () {56 vars in 1308 bytes} [Thu May 26 14:54:48 2022] GET /identity/v3/role_assignments?user.id=454431a7cdaf4374a1090691a2b55560&scope.system=all&role.id=adb2011fe8f74ecbbb94465253e76c1c => generated 510 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:54:48.801491 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5312b1-38a7-4d67-8a94-71172559336c None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:48.820098 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:48.927779 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5312b1-38a7-4d67-8a94-71172559336c None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:48.930931 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 184/366] 10.222.0.46 () {58 vars in 1282 bytes} [Thu May 26 14:54:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 14:54:48.936290 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e654826d-f709-4767-b553-5105ed77dbe0 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:48.977108 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23f4a1b-c93a-42ef-aee3-5bfe79ce3125 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:48.978360 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23f4a1b-c93a-42ef-aee3-5bfe79ce3125 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:48.980506 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23f4a1b-c93a-42ef-aee3-5bfe79ce3125 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:48.981087 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23f4a1b-c93a-42ef-aee3-5bfe79ce3125 None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:48.982150 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23f4a1b-c93a-42ef-aee3-5bfe79ce3125 None admin] RBAC: Adding query filter params (name=member) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:54:48.983621 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23f4a1b-c93a-42ef-aee3-5bfe79ce3125 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:54:48.987680 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:48.989190 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23f4a1b-c93a-42ef-aee3-5bfe79ce3125 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:54:49.006069 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 183/367] 10.222.0.46 () {56 vars in 1101 bytes} [Thu May 26 14:54:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:54:49.008072 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23f4a1b-c93a-42ef-aee3-5bfe79ce3125 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:49.131605 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e654826d-f709-4767-b553-5105ed77dbe0 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:49.155347 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:49.174039 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4980401-220e-47fa-9cc1-dafd0dae69d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:49.175298 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4980401-220e-47fa-9cc1-dafd0dae69d7 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:49.175870 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4980401-220e-47fa-9cc1-dafd0dae69d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:49.176362 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4980401-220e-47fa-9cc1-dafd0dae69d7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:49.189794 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4980401-220e-47fa-9cc1-dafd0dae69d7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 14:54:49.193991 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23f4a1b-c93a-42ef-aee3-5bfe79ce3125 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:49.199481 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-e4980401-220e-47fa-9cc1-dafd0dae69d7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 14:54:49.199481 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:54:49.199481 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:54:49.199481 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:54:49.199481 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:54:49.199481 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:54:49.199481 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:54:49.199481 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:54:49.199481 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:54:49.199481 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:54:49.199481 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:54:49.199481 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:54:49.199481 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 14:54:49.199481 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 14:54:49.199481 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 14:54:49.199481 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 14:54:49.200379 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 14:54:49.200379 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 14:54:49.200379 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 14:54:49.200379 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 14:54:49.200379 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:54:49.200379 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:54:49.200379 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:54:49.200379 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:54:49.200379 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:54:49.200379 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:54:49.200379 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:54:49.200379 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 26 14:54:49.200379 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:54:49.200379 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 26 14:54:49.200379 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:54:49.200379 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:54:49.201177 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:54:49.201177 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 26 14:54:49.201177 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:54:49.201177 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 26 14:54:49.201177 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 14:54:49.201177 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 14:54:49.201177 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 14:54:49.201177 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 14:54:49.201177 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 14:54:49.201177 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 14:54:49.201177 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 14:54:49.201177 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 26 14:54:49.203174 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 185/368] 10.222.0.46 () {56 vars in 1096 bytes} [Thu May 26 14:54:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:54:49.204218 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4980401-220e-47fa-9cc1-dafd0dae69d7 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:49.244053 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec65a68-6105-442c-9174-31308e3436c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:49.246455 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec65a68-6105-442c-9174-31308e3436c6 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:49.246883 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec65a68-6105-442c-9174-31308e3436c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:49.247370 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec65a68-6105-442c-9174-31308e3436c6 None admin] PATH_INFO: `/v3/projects/94ad8a7442da4d658f58d1543ae06493` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:49.249320 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec65a68-6105-442c-9174-31308e3436c6 None admin] RBAC: Authorizing `identity:get_project(project_id=94ad8a7442da4d658f58d1543ae06493)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:54:49.251717 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fec65a68-6105-442c-9174-31308e3436c6 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:54:49.255154 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 184/369] 10.222.0.46 () {56 vars in 1186 bytes} [Thu May 26 14:54:49 2022] GET /identity/v3/projects/94ad8a7442da4d658f58d1543ae06493 => generated 302 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:54:49.256443 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec65a68-6105-442c-9174-31308e3436c6 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:49.310905 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:49.339354 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4980401-220e-47fa-9cc1-dafd0dae69d7 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:49.379973 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bff17ef-509c-468d-b8f2-b8ac436a415e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:49.381339 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bff17ef-509c-468d-b8f2-b8ac436a415e None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:49.382805 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bff17ef-509c-468d-b8f2-b8ac436a415e None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:49.383235 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bff17ef-509c-468d-b8f2-b8ac436a415e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:49.391445 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fec65a68-6105-442c-9174-31308e3436c6 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:49.439975 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ce2752-fdeb-45d8-ade2-496de367b686 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:49.441368 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ce2752-fdeb-45d8-ade2-496de367b686 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:49.441864 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ce2752-fdeb-45d8-ade2-496de367b686 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:49.442651 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ce2752-fdeb-45d8-ade2-496de367b686 None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:49.443864 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ce2752-fdeb-45d8-ade2-496de367b686 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:54:49.444337 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ce2752-fdeb-45d8-ade2-496de367b686 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:54:49.446641 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ce2752-fdeb-45d8-ade2-496de367b686 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:54:49.460433 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 185/370] 10.222.0.46 () {56 vars in 1099 bytes} [Thu May 26 14:54:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:54:49.461701 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ce2752-fdeb-45d8-ade2-496de367b686 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:49.468477 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:49.478107 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bff17ef-509c-468d-b8f2-b8ac436a415e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:54:49.481095 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bff17ef-509c-468d-b8f2-b8ac436a415e None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:54:49.571272 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:49.595336 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 186/371] 10.222.0.46 () {58 vars in 1282 bytes} [Thu May 26 14:54:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 14:54:49.596463 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bff17ef-509c-468d-b8f2-b8ac436a415e None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:49.612960 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ce2752-fdeb-45d8-ade2-496de367b686 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:49.651651 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509fef87-75ea-4950-8419-02c3a50eaf00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:49.652517 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509fef87-75ea-4950-8419-02c3a50eaf00 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:49.652801 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509fef87-75ea-4950-8419-02c3a50eaf00 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:49.653072 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509fef87-75ea-4950-8419-02c3a50eaf00 None admin] PATH_INFO: `/v3/groups/4c84bf7a4c4f41da83d400606d236080` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:49.654271 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509fef87-75ea-4950-8419-02c3a50eaf00 None admin] RBAC: Authorizing `identity:get_group(group_id=4c84bf7a4c4f41da83d400606d236080)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:54:49.655727 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-509fef87-75ea-4950-8419-02c3a50eaf00 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:54:49.657739 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 186/372] 10.222.0.46 () {56 vars in 1180 bytes} [Thu May 26 14:54:49 2022] GET /identity/v3/groups/4c84bf7a4c4f41da83d400606d236080 => generated 226 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:54:49.658940 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509fef87-75ea-4950-8419-02c3a50eaf00 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:49.717242 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bff17ef-509c-468d-b8f2-b8ac436a415e None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:49.758485 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444105c6-9c1a-4007-90df-1bed57ed1d1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:49.759151 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444105c6-9c1a-4007-90df-1bed57ed1d1e None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:49.759436 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444105c6-9c1a-4007-90df-1bed57ed1d1e None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:49.759717 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444105c6-9c1a-4007-90df-1bed57ed1d1e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:49.782021 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:49.816696 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509fef87-75ea-4950-8419-02c3a50eaf00 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:49.817972 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:49.841437 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444105c6-9c1a-4007-90df-1bed57ed1d1e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:54:49.843091 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444105c6-9c1a-4007-90df-1bed57ed1d1e None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:54:49.853087 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b923f20b-6fd9-4b4d-8904-8a4e7293138c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:49.853980 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b923f20b-6fd9-4b4d-8904-8a4e7293138c None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:49.854262 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b923f20b-6fd9-4b4d-8904-8a4e7293138c None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:49.855111 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b923f20b-6fd9-4b4d-8904-8a4e7293138c None admin] PATH_INFO: `/v3/projects/94ad8a7442da4d658f58d1543ae06493` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:49.856303 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b923f20b-6fd9-4b4d-8904-8a4e7293138c None admin] RBAC: Authorizing `identity:get_project(project_id=94ad8a7442da4d658f58d1543ae06493)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:54:49.857815 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b923f20b-6fd9-4b4d-8904-8a4e7293138c None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:54:49.859820 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 187/373] 10.222.0.46 () {56 vars in 1186 bytes} [Thu May 26 14:54:49 2022] GET /identity/v3/projects/94ad8a7442da4d658f58d1543ae06493 => generated 302 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:54:49.860611 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b923f20b-6fd9-4b4d-8904-8a4e7293138c None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:49.955814 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 187/374] 10.222.0.46 () {58 vars in 1282 bytes} [Thu May 26 14:54:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 14:54:49.956779 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444105c6-9c1a-4007-90df-1bed57ed1d1e None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:49.970877 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b923f20b-6fd9-4b4d-8904-8a4e7293138c None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:50.019902 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca0c2ed-5fbe-40db-8ed3-6c7bd1a3a1b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:50.020570 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca0c2ed-5fbe-40db-8ed3-6c7bd1a3a1b2 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:50.020874 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca0c2ed-5fbe-40db-8ed3-6c7bd1a3a1b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:50.021256 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca0c2ed-5fbe-40db-8ed3-6c7bd1a3a1b2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:50.022230 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca0c2ed-5fbe-40db-8ed3-6c7bd1a3a1b2 None admin] RBAC: Adding query filter params (group.id=4c84bf7a4c4f41da83d400606d236080, role.id=160115b2143c4b389d39cd93f43e1d01, scope.project.id=94ad8a7442da4d658f58d1543ae06493) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:54:50.022544 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca0c2ed-5fbe-40db-8ed3-6c7bd1a3a1b2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:54:50.024129 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca0c2ed-5fbe-40db-8ed3-6c7bd1a3a1b2 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:54:50.031353 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 188/375] 10.222.0.46 () {56 vars in 1376 bytes} [Thu May 26 14:54:49 2022] GET /identity/v3/role_assignments?group.id=4c84bf7a4c4f41da83d400606d236080&scope.project.id=94ad8a7442da4d658f58d1543ae06493&role.id=160115b2143c4b389d39cd93f43e1d01 => generated 260 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:54:50.032172 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca0c2ed-5fbe-40db-8ed3-6c7bd1a3a1b2 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:50.035079 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:50.049253 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:50.072014 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444105c6-9c1a-4007-90df-1bed57ed1d1e None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:50.114313 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98df2602-80d1-49df-9b73-4df57c9ee890 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:50.115274 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98df2602-80d1-49df-9b73-4df57c9ee890 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:50.115621 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98df2602-80d1-49df-9b73-4df57c9ee890 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:50.115909 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98df2602-80d1-49df-9b73-4df57c9ee890 None admin] PATH_INFO: `/v3/groups/4c84bf7a4c4f41da83d400606d236080` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:50.117210 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98df2602-80d1-49df-9b73-4df57c9ee890 None admin] RBAC: Authorizing `identity:get_group(group_id=4c84bf7a4c4f41da83d400606d236080)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:54:50.118877 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98df2602-80d1-49df-9b73-4df57c9ee890 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:54:50.120886 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 188/376] 10.222.0.46 () {56 vars in 1180 bytes} [Thu May 26 14:54:49 2022] GET /identity/v3/groups/4c84bf7a4c4f41da83d400606d236080 => generated 226 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:54:50.127360 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98df2602-80d1-49df-9b73-4df57c9ee890 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:50.158485 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca0c2ed-5fbe-40db-8ed3-6c7bd1a3a1b2 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:50.199533 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe2cc59-c2a6-4d35-8ae2-e5393991ec48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:50.200423 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe2cc59-c2a6-4d35-8ae2-e5393991ec48 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:50.200837 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe2cc59-c2a6-4d35-8ae2-e5393991ec48 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:50.201117 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe2cc59-c2a6-4d35-8ae2-e5393991ec48 None admin] PATH_INFO: `/v3/projects/064fc3c7af494c988e44e426f21b1c78` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:50.202554 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe2cc59-c2a6-4d35-8ae2-e5393991ec48 None admin] RBAC: Authorizing `identity:get_project(project_id=064fc3c7af494c988e44e426f21b1c78)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:54:50.204065 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe2cc59-c2a6-4d35-8ae2-e5393991ec48 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:54:50.206008 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 189/377] 10.222.0.46 () {56 vars in 1186 bytes} [Thu May 26 14:54:50 2022] GET /identity/v3/projects/064fc3c7af494c988e44e426f21b1c78 => generated 344 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:54:50.212337 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe2cc59-c2a6-4d35-8ae2-e5393991ec48 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:50.250890 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98df2602-80d1-49df-9b73-4df57c9ee890 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:50.292174 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c87060c-4c49-4750-b0fe-98990008c55b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:50.292899 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c87060c-4c49-4750-b0fe-98990008c55b None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:50.293190 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c87060c-4c49-4750-b0fe-98990008c55b None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:50.293504 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c87060c-4c49-4750-b0fe-98990008c55b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:50.294337 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c87060c-4c49-4750-b0fe-98990008c55b None admin] RBAC: Adding query filter params (group.id=4c84bf7a4c4f41da83d400606d236080, role.id=d4088d1845a24fbd9bd8975ce0b1d3d0, scope.project.id=94ad8a7442da4d658f58d1543ae06493) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:54:50.294660 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c87060c-4c49-4750-b0fe-98990008c55b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:54:50.295304 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:50.297342 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c87060c-4c49-4750-b0fe-98990008c55b None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:54:50.304691 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 189/378] 10.222.0.46 () {56 vars in 1376 bytes} [Thu May 26 14:54:50 2022] GET /identity/v3/role_assignments?group.id=4c84bf7a4c4f41da83d400606d236080&scope.project.id=94ad8a7442da4d658f58d1543ae06493&role.id=d4088d1845a24fbd9bd8975ce0b1d3d0 => generated 260 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:54:50.336988 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe2cc59-c2a6-4d35-8ae2-e5393991ec48 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:50.390647 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52089b51-ee7a-42dc-a186-fd6adf15a05e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:50.391801 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52089b51-ee7a-42dc-a186-fd6adf15a05e None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:50.392220 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52089b51-ee7a-42dc-a186-fd6adf15a05e None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:50.392505 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52089b51-ee7a-42dc-a186-fd6adf15a05e None admin] PATH_INFO: `/v3/groups/7757649a149e40c398b4898ac4f6d4a3` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:50.400032 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52089b51-ee7a-42dc-a186-fd6adf15a05e None admin] RBAC: Authorizing `identity:get_group(group_id=7757649a149e40c398b4898ac4f6d4a3)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:54:50.402123 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52089b51-ee7a-42dc-a186-fd6adf15a05e None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:54:50.404312 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 190/379] 10.222.0.46 () {56 vars in 1180 bytes} [Thu May 26 14:54:50 2022] GET /identity/v3/groups/7757649a149e40c398b4898ac4f6d4a3 => generated 229 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:54:50.410535 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c87060c-4c49-4750-b0fe-98990008c55b None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:50.432994 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:50.487484 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa3c195-cc52-4207-8fc6-94433a756285 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:50.488057 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa3c195-cc52-4207-8fc6-94433a756285 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:50.488513 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa3c195-cc52-4207-8fc6-94433a756285 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:50.490229 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 191/380] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 14:54:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 14:54:50.504532 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6564886e-83eb-49ce-9b62-9c16748e8343 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:50.505137 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6564886e-83eb-49ce-9b62-9c16748e8343 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:50.506129 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6564886e-83eb-49ce-9b62-9c16748e8343 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:50.535328 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-6564886e-83eb-49ce-9b62-9c16748e8343 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:54:50.537009 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-6564886e-83eb-49ce-9b62-9c16748e8343 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:54:50.554011 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c87060c-4c49-4750-b0fe-98990008c55b None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:50.598567 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1913fcdb-d539-4e1b-8cb4-108fdf6855b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:50.599332 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1913fcdb-d539-4e1b-8cb4-108fdf6855b0 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:50.599521 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1913fcdb-d539-4e1b-8cb4-108fdf6855b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:50.599837 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1913fcdb-d539-4e1b-8cb4-108fdf6855b0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:50.600696 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1913fcdb-d539-4e1b-8cb4-108fdf6855b0 None admin] RBAC: Adding query filter params (group.id=7757649a149e40c398b4898ac4f6d4a3, role.id=a2e551773cbd4df084020676865b8222, scope.project.id=064fc3c7af494c988e44e426f21b1c78) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:54:50.601015 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1913fcdb-d539-4e1b-8cb4-108fdf6855b0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:54:50.602790 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1913fcdb-d539-4e1b-8cb4-108fdf6855b0 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:54:50.613872 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 190/381] 10.222.0.46 () {56 vars in 1376 bytes} [Thu May 26 14:54:50 2022] GET /identity/v3/role_assignments?group.id=7757649a149e40c398b4898ac4f6d4a3&scope.project.id=064fc3c7af494c988e44e426f21b1c78&role.id=a2e551773cbd4df084020676865b8222 => generated 260 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:54:50.649313 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 192/382] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 14:54:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 14:54:50.673603 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1913fcdb-d539-4e1b-8cb4-108fdf6855b0 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:50.787090 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:50.825490 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1913fcdb-d539-4e1b-8cb4-108fdf6855b0 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:50.871940 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf49410-4983-4b1d-b5ca-66963e38fbd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:50.872768 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf49410-4983-4b1d-b5ca-66963e38fbd7 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:50.873052 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf49410-4983-4b1d-b5ca-66963e38fbd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:50.873330 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf49410-4983-4b1d-b5ca-66963e38fbd7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:50.887137 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf49410-4983-4b1d-b5ca-66963e38fbd7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 14:54:50.895120 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-baf49410-4983-4b1d-b5ca-66963e38fbd7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 26 14:54:50.895120 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:54:50.895120 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:54:50.895120 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:54:50.895120 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:54:50.895120 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:54:50.895120 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:54:50.895120 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:54:50.895120 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:54:50.895120 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:54:50.895120 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:54:50.895120 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:54:50.895120 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 14:54:50.895120 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 14:54:50.895120 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 14:54:50.895120 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 14:54:50.895999 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 14:54:50.895999 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 14:54:50.895999 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 14:54:50.895999 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 14:54:50.895999 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:54:50.895999 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:54:50.895999 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:54:50.895999 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:54:50.895999 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:54:50.895999 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:54:50.895999 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:54:50.895999 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 26 14:54:50.895999 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:54:50.895999 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 26 14:54:50.895999 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:54:50.895999 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:54:50.896849 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:54:50.896849 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 26 14:54:50.896849 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:54:50.896849 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 26 14:54:50.896849 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 14:54:50.896849 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 14:54:50.896849 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 14:54:50.896849 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 14:54:50.896849 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 14:54:50.896849 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 14:54:50.896849 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 26 14:54:50.896849 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 26 14:54:50.896849 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 191/383] 10.222.0.46 () {56 vars in 1099 bytes} [Thu May 26 14:54:50 2022] GET /identity/v3/roles/member => generated 84 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:54:50.904777 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6564886e-83eb-49ce-9b62-9c16748e8343 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:50.982239 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:51.022472 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6564886e-83eb-49ce-9b62-9c16748e8343 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:51.067078 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27145d73-2fff-4863-94bf-a191d20b1e61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:51.067790 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27145d73-2fff-4863-94bf-a191d20b1e61 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:51.068075 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27145d73-2fff-4863-94bf-a191d20b1e61 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:51.068354 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27145d73-2fff-4863-94bf-a191d20b1e61 None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:51.069144 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27145d73-2fff-4863-94bf-a191d20b1e61 None admin] RBAC: Adding query filter params (name=member) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:54:51.069473 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27145d73-2fff-4863-94bf-a191d20b1e61 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:54:51.070920 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27145d73-2fff-4863-94bf-a191d20b1e61 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:54:51.081704 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 193/384] 10.222.0.46 () {56 vars in 1101 bytes} [Thu May 26 14:54:50 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:54:51.090612 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf49410-4983-4b1d-b5ca-66963e38fbd7 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:51.252710 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf49410-4983-4b1d-b5ca-66963e38fbd7 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:51.288326 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c2d56b-9199-4a25-a8e2-72de9524f7ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:51.289226 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c2d56b-9199-4a25-a8e2-72de9524f7ae None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:51.289534 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c2d56b-9199-4a25-a8e2-72de9524f7ae None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:51.289813 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c2d56b-9199-4a25-a8e2-72de9524f7ae None admin] PATH_INFO: `/v3/groups/4c84bf7a4c4f41da83d400606d236080` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:51.291044 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c2d56b-9199-4a25-a8e2-72de9524f7ae None admin] RBAC: Authorizing `identity:get_group(group_id=4c84bf7a4c4f41da83d400606d236080)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:54:51.292548 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00c2d56b-9199-4a25-a8e2-72de9524f7ae None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:54:51.294549 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 192/385] 10.222.0.46 () {56 vars in 1180 bytes} [Thu May 26 14:54:51 2022] GET /identity/v3/groups/4c84bf7a4c4f41da83d400606d236080 => generated 226 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:54:51.299950 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27145d73-2fff-4863-94bf-a191d20b1e61 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:51.311762 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0912277b-a5c0-469c-b79e-1774774dbb29 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:51.311762 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0912277b-a5c0-469c-b79e-1774774dbb29 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:51.311762 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0912277b-a5c0-469c-b79e-1774774dbb29 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:51.313074 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 193/386] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 14:54:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 14:54:51.318079 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aadd56a-8df1-4730-84ae-9691f878a0de None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:51.318418 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aadd56a-8df1-4730-84ae-9691f878a0de None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:51.318696 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aadd56a-8df1-4730-84ae-9691f878a0de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:51.327988 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:51.340875 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-2aadd56a-8df1-4730-84ae-9691f878a0de None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:54:51.341771 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-2aadd56a-8df1-4730-84ae-9691f878a0de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:54:51.442103 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 194/387] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 14:54:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 14:54:51.461150 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27145d73-2fff-4863-94bf-a191d20b1e61 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:51.479317 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aadd56a-8df1-4730-84ae-9691f878a0de None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:51.509356 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:51.521735 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda83bbc-d1f1-4712-b001-ec48af05c140 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:51.522644 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda83bbc-d1f1-4712-b001-ec48af05c140 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:51.523035 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda83bbc-d1f1-4712-b001-ec48af05c140 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:51.523313 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda83bbc-d1f1-4712-b001-ec48af05c140 None admin] PATH_INFO: `/v3/projects/e474420e5c124a3f86383fce1ef4c4fb` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:51.525147 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda83bbc-d1f1-4712-b001-ec48af05c140 None admin] RBAC: Authorizing `identity:get_project(project_id=e474420e5c124a3f86383fce1ef4c4fb)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:54:51.527342 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda83bbc-d1f1-4712-b001-ec48af05c140 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:54:51.530796 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 194/388] 10.222.0.46 () {56 vars in 1186 bytes} [Thu May 26 14:54:51 2022] GET /identity/v3/projects/e474420e5c124a3f86383fce1ef4c4fb => generated 298 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:54:51.534856 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda83bbc-d1f1-4712-b001-ec48af05c140 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:51.662229 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aadd56a-8df1-4730-84ae-9691f878a0de None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:51.702469 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda83bbc-d1f1-4712-b001-ec48af05c140 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:51.704740 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42945b44-17a4-48e2-89c4-99041e3506f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:51.706544 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42945b44-17a4-48e2-89c4-99041e3506f5 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:51.706827 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42945b44-17a4-48e2-89c4-99041e3506f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:51.707102 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42945b44-17a4-48e2-89c4-99041e3506f5 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:51.709107 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:51.722257 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42945b44-17a4-48e2-89c4-99041e3506f5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 14:54:51.734111 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-42945b44-17a4-48e2-89c4-99041e3506f5 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 14:54:51.734111 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:54:51.734111 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:54:51.734111 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:54:51.734111 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:54:51.734111 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:54:51.734111 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:54:51.734111 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:54:51.734111 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:54:51.734111 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:54:51.734111 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:54:51.734111 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:54:51.734111 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 14:54:51.734111 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 14:54:51.734111 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 14:54:51.734111 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 14:54:51.735255 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 14:54:51.735255 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 14:54:51.735255 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 14:54:51.735255 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 14:54:51.735255 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:54:51.735255 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:54:51.735255 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:54:51.735255 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:54:51.735255 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:54:51.735255 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:54:51.735255 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:54:51.735255 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 26 14:54:51.735255 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:54:51.735255 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 26 14:54:51.735255 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:54:51.735255 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:54:51.736208 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:54:51.736208 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 26 14:54:51.736208 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:54:51.736208 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 26 14:54:51.736208 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 14:54:51.736208 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 14:54:51.736208 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 14:54:51.736208 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 14:54:51.736208 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 14:54:51.736208 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 14:54:51.736208 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 14:54:51.736208 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 26 14:54:51.736988 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 195/389] 10.222.0.46 () {56 vars in 1114 bytes} [Thu May 26 14:54:51 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 14:54:51.746526 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42945b44-17a4-48e2-89c4-99041e3506f5 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:51.756880 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f21792-616e-4b75-b95f-920025cf89bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:51.766574 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f21792-616e-4b75-b95f-920025cf89bc None admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:51.766795 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f21792-616e-4b75-b95f-920025cf89bc None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:51.767080 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f21792-616e-4b75-b95f-920025cf89bc None admin] PATH_INFO: `/v3/projects/e474420e5c124a3f86383fce1ef4c4fb/groups/4c84bf7a4c4f41da83d400606d236080/roles/d4088d1845a24fbd9bd8975ce0b1d3d0` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:51.772120 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f21792-616e-4b75-b95f-920025cf89bc None admin] RBAC: Authorizing `identity:create_grant(project_id=e474420e5c124a3f86383fce1ef4c4fb, group_id=4c84bf7a4c4f41da83d400606d236080, role_id=d4088d1845a24fbd9bd8975ce0b1d3d0)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:54:51.775202 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f21792-616e-4b75-b95f-920025cf89bc None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:54:51.801574 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 195/390] 10.222.0.46 () {58 vars in 1442 bytes} [Thu May 26 14:54:51 2022] PUT /identity/v3/projects/e474420e5c124a3f86383fce1ef4c4fb/groups/4c84bf7a4c4f41da83d400606d236080/roles/d4088d1845a24fbd9bd8975ce0b1d3d0 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 14:54:51.882014 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42945b44-17a4-48e2-89c4-99041e3506f5 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:51.918908 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70dd62b7-3384-47b2-8d3e-d249bcd2ee95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:51.919861 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dd62b7-3384-47b2-8d3e-d249bcd2ee95 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:51.920294 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dd62b7-3384-47b2-8d3e-d249bcd2ee95 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:51.920720 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dd62b7-3384-47b2-8d3e-d249bcd2ee95 None admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:51.921661 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70dd62b7-3384-47b2-8d3e-d249bcd2ee95 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:54:51.922131 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70dd62b7-3384-47b2-8d3e-d249bcd2ee95 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:54:51.923704 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70dd62b7-3384-47b2-8d3e-d249bcd2ee95 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:54:51.937554 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 196/391] 10.222.0.46 () {56 vars in 1111 bytes} [Thu May 26 14:54:51 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:54:51.940378 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f21792-616e-4b75-b95f-920025cf89bc None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:52.003379 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:52.054842 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f21792-616e-4b75-b95f-920025cf89bc None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:52.091443 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4afb8ee-b1c0-426d-a4fe-76e6b2097569 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:52.092569 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4afb8ee-b1c0-426d-a4fe-76e6b2097569 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:52.092999 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4afb8ee-b1c0-426d-a4fe-76e6b2097569 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:52.093419 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4afb8ee-b1c0-426d-a4fe-76e6b2097569 None admin] PATH_INFO: `/v3/groups/4c84bf7a4c4f41da83d400606d236080` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:52.094813 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4afb8ee-b1c0-426d-a4fe-76e6b2097569 None admin] RBAC: Authorizing `identity:get_group(group_id=4c84bf7a4c4f41da83d400606d236080)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:54:52.096421 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4afb8ee-b1c0-426d-a4fe-76e6b2097569 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:54:52.098287 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 196/392] 10.222.0.46 () {56 vars in 1180 bytes} [Thu May 26 14:54:51 2022] GET /identity/v3/groups/4c84bf7a4c4f41da83d400606d236080 => generated 226 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:54:52.106500 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70dd62b7-3384-47b2-8d3e-d249bcd2ee95 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:52.168281 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:52.213428 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70dd62b7-3384-47b2-8d3e-d249bcd2ee95 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:52.249129 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001f86f4-7008-4bee-8b9a-c6aa087e4410 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:52.250299 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001f86f4-7008-4bee-8b9a-c6aa087e4410 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:52.250729 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001f86f4-7008-4bee-8b9a-c6aa087e4410 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:52.251150 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001f86f4-7008-4bee-8b9a-c6aa087e4410 None admin] PATH_INFO: `/v3/projects/e474420e5c124a3f86383fce1ef4c4fb` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:52.252640 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001f86f4-7008-4bee-8b9a-c6aa087e4410 None admin] RBAC: Authorizing `identity:get_project(project_id=e474420e5c124a3f86383fce1ef4c4fb)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:54:52.254320 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001f86f4-7008-4bee-8b9a-c6aa087e4410 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:54:52.256569 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 197/393] 10.222.0.46 () {56 vars in 1186 bytes} [Thu May 26 14:54:52 2022] GET /identity/v3/projects/e474420e5c124a3f86383fce1ef4c4fb => generated 298 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:54:52.262655 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4afb8ee-b1c0-426d-a4fe-76e6b2097569 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:52.382932 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4afb8ee-b1c0-426d-a4fe-76e6b2097569 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:52.408020 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:52.420470 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ec9ab7-a31f-4dc0-951f-dfb618c9ddf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:52.421606 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ec9ab7-a31f-4dc0-951f-dfb618c9ddf4 None admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:52.422030 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ec9ab7-a31f-4dc0-951f-dfb618c9ddf4 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:52.422447 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ec9ab7-a31f-4dc0-951f-dfb618c9ddf4 None admin] PATH_INFO: `/v3/projects/e474420e5c124a3f86383fce1ef4c4fb/groups/4c84bf7a4c4f41da83d400606d236080/roles/160115b2143c4b389d39cd93f43e1d01` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:52.424799 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ec9ab7-a31f-4dc0-951f-dfb618c9ddf4 None admin] RBAC: Authorizing `identity:create_grant(project_id=e474420e5c124a3f86383fce1ef4c4fb, group_id=4c84bf7a4c4f41da83d400606d236080, role_id=160115b2143c4b389d39cd93f43e1d01)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:54:52.426602 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ec9ab7-a31f-4dc0-951f-dfb618c9ddf4 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:54:52.444966 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 197/394] 10.222.0.46 () {58 vars in 1442 bytes} [Thu May 26 14:54:52 2022] PUT /identity/v3/projects/e474420e5c124a3f86383fce1ef4c4fb/groups/4c84bf7a4c4f41da83d400606d236080/roles/160115b2143c4b389d39cd93f43e1d01 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 14:54:53.253763 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fd2019-2e7a-4640-857a-093dd93dc6bd None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:53.254636 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fd2019-2e7a-4640-857a-093dd93dc6bd None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:53.255074 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3fd2019-2e7a-4640-857a-093dd93dc6bd None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:53.257125 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 198/395] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 14:54:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 14:54:53.264185 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1626864f-d447-4271-9203-05fb4d8c3af3 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:53.264935 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1626864f-d447-4271-9203-05fb4d8c3af3 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:53.265357 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1626864f-d447-4271-9203-05fb4d8c3af3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:53.283116 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-1626864f-d447-4271-9203-05fb4d8c3af3 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:54:53.284521 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-1626864f-d447-4271-9203-05fb4d8c3af3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:54:53.377596 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 198/396] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 14:54:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 14:54:53.397768 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fd2019-2e7a-4640-857a-093dd93dc6bd None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:53.514223 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3fd2019-2e7a-4640-857a-093dd93dc6bd None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:53.557835 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:53.575444 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15eed3c-2705-4407-8875-1afef81389ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:53.576257 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15eed3c-2705-4407-8875-1afef81389ef None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:53.576545 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15eed3c-2705-4407-8875-1afef81389ef None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:53.576826 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15eed3c-2705-4407-8875-1afef81389ef None admin] PATH_INFO: `/v3/roles/member` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:53.583101 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02939a5a-3b76-488a-820a-e1b3ffb13f6b None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:53.583656 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02939a5a-3b76-488a-820a-e1b3ffb13f6b None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:53.584135 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02939a5a-3b76-488a-820a-e1b3ffb13f6b None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:53.585984 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 199/397] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 14:54:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 14:54:53.591780 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5620925d-c043-41b8-995a-9bd3de4156ae None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:53.592319 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5620925d-c043-41b8-995a-9bd3de4156ae None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:53.592815 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5620925d-c043-41b8-995a-9bd3de4156ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:53.595776 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15eed3c-2705-4407-8875-1afef81389ef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 14:54:53.603859 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-b15eed3c-2705-4407-8875-1afef81389ef None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 26 14:54:53.603859 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:54:53.603859 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:54:53.603859 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:54:53.603859 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:54:53.603859 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:54:53.603859 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:54:53.603859 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:54:53.603859 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:54:53.603859 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:54:53.603859 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:54:53.603859 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:54:53.603859 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 14:54:53.603859 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 14:54:53.603859 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 14:54:53.603859 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 14:54:53.604888 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 14:54:53.604888 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 14:54:53.604888 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 14:54:53.604888 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 14:54:53.604888 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:54:53.604888 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:54:53.604888 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:54:53.604888 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:54:53.604888 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:54:53.604888 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:54:53.604888 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:54:53.604888 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 26 14:54:53.604888 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:54:53.604888 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 26 14:54:53.604888 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:54:53.604888 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:54:53.605707 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:54:53.605707 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 26 14:54:53.605707 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:54:53.605707 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 26 14:54:53.605707 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 14:54:53.605707 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 14:54:53.605707 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 14:54:53.605707 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 14:54:53.605707 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 14:54:53.605707 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 14:54:53.605707 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 26 14:54:53.605707 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 26 14:54:53.613942 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 199/398] 10.222.0.46 () {56 vars in 1099 bytes} [Thu May 26 14:54:53 2022] GET /identity/v3/roles/member => generated 84 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:54:53.621512 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15eed3c-2705-4407-8875-1afef81389ef None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:53.625797 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-5620925d-c043-41b8-995a-9bd3de4156ae None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:54:53.627409 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-5620925d-c043-41b8-995a-9bd3de4156ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:54:53.710325 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 200/399] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 14:54:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 14:54:53.717610 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c60eb7b-7206-4cd4-88fb-2336b014b3f4 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:53.717610 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c60eb7b-7206-4cd4-88fb-2336b014b3f4 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:53.717610 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c60eb7b-7206-4cd4-88fb-2336b014b3f4 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:53.717610 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 201/400] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 14:54:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 14:54:53.719438 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d82a708-366d-4df7-b5e6-fcdff5d7b3db None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:53.719733 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d82a708-366d-4df7-b5e6-fcdff5d7b3db None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:53.720015 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d82a708-366d-4df7-b5e6-fcdff5d7b3db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:53.740136 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-6d82a708-366d-4df7-b5e6-fcdff5d7b3db None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:54:53.740728 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-6d82a708-366d-4df7-b5e6-fcdff5d7b3db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:54:53.775191 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15eed3c-2705-4407-8875-1afef81389ef None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:53.799108 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:53.820171 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c4c8e6-37f9-4659-b32a-9f6858c3e99e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:53.820952 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c4c8e6-37f9-4659-b32a-9f6858c3e99e None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:53.821347 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c4c8e6-37f9-4659-b32a-9f6858c3e99e None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:53.821725 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c4c8e6-37f9-4659-b32a-9f6858c3e99e None admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:53.822540 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c4c8e6-37f9-4659-b32a-9f6858c3e99e None admin] RBAC: Adding query filter params (name=member) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:54:53.822956 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c4c8e6-37f9-4659-b32a-9f6858c3e99e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:54:53.825139 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c4c8e6-37f9-4659-b32a-9f6858c3e99e None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:54:53.842702 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 200/401] 10.222.0.46 () {56 vars in 1101 bytes} [Thu May 26 14:54:53 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 14:54:53.843653 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c4c8e6-37f9-4659-b32a-9f6858c3e99e None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:53.844551 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 202/402] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 14:54:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 14:54:53.849005 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d82a708-366d-4df7-b5e6-fcdff5d7b3db None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:53.914441 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:53.972577 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c4c8e6-37f9-4659-b32a-9f6858c3e99e None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:53.974399 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d82a708-366d-4df7-b5e6-fcdff5d7b3db None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:54.010644 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfcab37-cadf-464c-9714-ae9f3b8a40bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:54.011780 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379cc6a0-60f8-4ca3-92ca-6bd917b52e51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:54.012053 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfcab37-cadf-464c-9714-ae9f3b8a40bf None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:54.012509 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfcab37-cadf-464c-9714-ae9f3b8a40bf None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:54.012785 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379cc6a0-60f8-4ca3-92ca-6bd917b52e51 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:54.013133 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfcab37-cadf-464c-9714-ae9f3b8a40bf None admin] PATH_INFO: `/v3/groups/4c84bf7a4c4f41da83d400606d236080` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:54.013244 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379cc6a0-60f8-4ca3-92ca-6bd917b52e51 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:54.013498 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379cc6a0-60f8-4ca3-92ca-6bd917b52e51 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:54.014793 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddfcab37-cadf-464c-9714-ae9f3b8a40bf None admin] RBAC: Authorizing `identity:get_group(group_id=4c84bf7a4c4f41da83d400606d236080)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:54:54.016452 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddfcab37-cadf-464c-9714-ae9f3b8a40bf None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:54:54.018629 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 203/403] 10.222.0.46 () {56 vars in 1180 bytes} [Thu May 26 14:54:53 2022] GET /identity/v3/groups/4c84bf7a4c4f41da83d400606d236080 => generated 226 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:54:54.019663 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfcab37-cadf-464c-9714-ae9f3b8a40bf None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:54.030112 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379cc6a0-60f8-4ca3-92ca-6bd917b52e51 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 14:54:54.038480 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-379cc6a0-60f8-4ca3-92ca-6bd917b52e51 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 14:54:54.038480 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:54:54.038480 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:54:54.038480 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:54:54.038480 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:54:54.038480 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:54:54.038480 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:54:54.038480 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:54:54.038480 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:54:54.038480 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:54:54.038480 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:54:54.038480 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:54:54.038480 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 14:54:54.038480 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 14:54:54.038480 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 14:54:54.038480 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 14:54:54.039361 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 14:54:54.039361 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 14:54:54.039361 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 14:54:54.039361 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 14:54:54.039361 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:54:54.039361 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:54:54.039361 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:54:54.039361 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:54:54.039361 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:54:54.039361 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:54:54.039361 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:54:54.039361 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 26 14:54:54.039361 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:54:54.039361 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 26 14:54:54.039361 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:54:54.039361 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:54:54.040173 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:54:54.040173 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 26 14:54:54.040173 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:54:54.040173 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 26 14:54:54.040173 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 14:54:54.040173 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 14:54:54.040173 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 14:54:54.040173 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 14:54:54.040173 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 14:54:54.040173 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 14:54:54.040173 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 14:54:54.040173 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 26 14:54:54.044990 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 201/404] 10.222.0.46 () {56 vars in 1096 bytes} [Thu May 26 14:54:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:54:54.046239 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379cc6a0-60f8-4ca3-92ca-6bd917b52e51 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:54.067643 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:54.127562 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfcab37-cadf-464c-9714-ae9f3b8a40bf None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:54.158373 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379cc6a0-60f8-4ca3-92ca-6bd917b52e51 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:54.163786 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d066e4c-08a0-4178-99e4-b986a485914a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:54.164856 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d066e4c-08a0-4178-99e4-b986a485914a None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:54.165346 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d066e4c-08a0-4178-99e4-b986a485914a None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:54.165753 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:54.166064 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d066e4c-08a0-4178-99e4-b986a485914a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:54.177442 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d066e4c-08a0-4178-99e4-b986a485914a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 14:54:54.186630 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-8d066e4c-08a0-4178-99e4-b986a485914a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 14:54:54.186630 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:54:54.186630 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:54:54.186630 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:54:54.186630 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:54:54.186630 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:54:54.186630 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:54:54.186630 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:54:54.186630 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:54:54.186630 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:54:54.186630 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:54:54.186630 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:54:54.186630 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 14:54:54.186630 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 14:54:54.186630 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 14:54:54.186630 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 14:54:54.187518 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 14:54:54.187518 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 14:54:54.187518 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 14:54:54.187518 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 14:54:54.187518 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:54:54.187518 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:54:54.187518 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:54:54.187518 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:54:54.187518 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:54:54.187518 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:54:54.187518 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:54:54.187518 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 26 14:54:54.187518 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:54:54.187518 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 26 14:54:54.187518 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:54:54.187518 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:54:54.188303 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:54:54.188303 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 26 14:54:54.188303 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:54:54.188303 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 26 14:54:54.188303 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 14:54:54.188303 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 14:54:54.188303 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 14:54:54.188303 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 14:54:54.188303 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 14:54:54.188303 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 14:54:54.188303 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 14:54:54.188303 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 26 14:54:54.190663 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 204/405] 10.222.0.46 () {56 vars in 1114 bytes} [Thu May 26 14:54:54 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:54:54.191643 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d066e4c-08a0-4178-99e4-b986a485914a None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:54.201293 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca4be43-e01c-4b64-80e4-c474dec21c14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:54.202204 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca4be43-e01c-4b64-80e4-c474dec21c14 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:54.202487 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca4be43-e01c-4b64-80e4-c474dec21c14 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:54.202763 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca4be43-e01c-4b64-80e4-c474dec21c14 None admin] PATH_INFO: `/v3/projects/94ad8a7442da4d658f58d1543ae06493` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:54.203948 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca4be43-e01c-4b64-80e4-c474dec21c14 None admin] RBAC: Authorizing `identity:get_project(project_id=94ad8a7442da4d658f58d1543ae06493)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:54:54.205482 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca4be43-e01c-4b64-80e4-c474dec21c14 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:54:54.207278 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 202/406] 10.222.0.46 () {56 vars in 1186 bytes} [Thu May 26 14:54:54 2022] GET /identity/v3/projects/94ad8a7442da4d658f58d1543ae06493 => generated 302 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:54:54.212632 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca4be43-e01c-4b64-80e4-c474dec21c14 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:54.295590 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d066e4c-08a0-4178-99e4-b986a485914a None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:54.315908 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:54.316549 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca4be43-e01c-4b64-80e4-c474dec21c14 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:54.330745 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24666d3e-bb0f-41bf-9081-c313ca4f0210 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:54.331812 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24666d3e-bb0f-41bf-9081-c313ca4f0210 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:54.332293 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24666d3e-bb0f-41bf-9081-c313ca4f0210 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:54.332781 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24666d3e-bb0f-41bf-9081-c313ca4f0210 None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:54.333922 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24666d3e-bb0f-41bf-9081-c313ca4f0210 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:54:54.334596 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24666d3e-bb0f-41bf-9081-c313ca4f0210 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:54:54.336232 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24666d3e-bb0f-41bf-9081-c313ca4f0210 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:54:54.347038 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 205/407] 10.222.0.46 () {56 vars in 1099 bytes} [Thu May 26 14:54:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:54:54.348123 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24666d3e-bb0f-41bf-9081-c313ca4f0210 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:54.357277 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7642a273-6232-4676-bde2-932b511b41ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:54.358156 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7642a273-6232-4676-bde2-932b511b41ca None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:54.358372 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7642a273-6232-4676-bde2-932b511b41ca None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:54.358638 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7642a273-6232-4676-bde2-932b511b41ca None admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:54.359449 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7642a273-6232-4676-bde2-932b511b41ca None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:54:54.359762 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7642a273-6232-4676-bde2-932b511b41ca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:54:54.361134 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7642a273-6232-4676-bde2-932b511b41ca None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:54:54.370084 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 203/408] 10.222.0.46 () {56 vars in 1111 bytes} [Thu May 26 14:54:54 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:54:54.375368 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7642a273-6232-4676-bde2-932b511b41ca None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:54.413001 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:54.480222 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24666d3e-bb0f-41bf-9081-c313ca4f0210 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:54.487173 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7642a273-6232-4676-bde2-932b511b41ca None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:54.522264 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f2e6a0-6877-4612-966b-f522f52463c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:54.523130 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f2e6a0-6877-4612-966b-f522f52463c1 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:54.523419 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f2e6a0-6877-4612-966b-f522f52463c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:54.523700 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81f2e6a0-6877-4612-966b-f522f52463c1 None admin] PATH_INFO: `/v3/groups/7757649a149e40c398b4898ac4f6d4a3` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:54.524874 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f2e6a0-6877-4612-966b-f522f52463c1 None admin] RBAC: Authorizing `identity:get_group(group_id=7757649a149e40c398b4898ac4f6d4a3)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:54:54.525382 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349a2460-332d-4797-904f-bf75ddf7da40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:54.526950 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81f2e6a0-6877-4612-966b-f522f52463c1 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:54:54.527244 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349a2460-332d-4797-904f-bf75ddf7da40 None admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:54.527753 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349a2460-332d-4797-904f-bf75ddf7da40 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:54.528219 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349a2460-332d-4797-904f-bf75ddf7da40 None admin] PATH_INFO: `/v3/projects/94ad8a7442da4d658f58d1543ae06493/groups/4c84bf7a4c4f41da83d400606d236080/roles/d4088d1845a24fbd9bd8975ce0b1d3d0` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:54.530513 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349a2460-332d-4797-904f-bf75ddf7da40 None admin] RBAC: Authorizing `identity:create_grant(project_id=94ad8a7442da4d658f58d1543ae06493, group_id=4c84bf7a4c4f41da83d400606d236080, role_id=d4088d1845a24fbd9bd8975ce0b1d3d0)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:54:54.532257 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349a2460-332d-4797-904f-bf75ddf7da40 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:54:54.533501 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 204/409] 10.222.0.46 () {56 vars in 1180 bytes} [Thu May 26 14:54:54 2022] GET /identity/v3/groups/7757649a149e40c398b4898ac4f6d4a3 => generated 229 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:54:54.534476 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f2e6a0-6877-4612-966b-f522f52463c1 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:54.552504 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 206/410] 10.222.0.46 () {58 vars in 1442 bytes} [Thu May 26 14:54:54 2022] PUT /identity/v3/projects/94ad8a7442da4d658f58d1543ae06493/groups/4c84bf7a4c4f41da83d400606d236080/roles/d4088d1845a24fbd9bd8975ce0b1d3d0 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 14:54:54.553763 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349a2460-332d-4797-904f-bf75ddf7da40 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:54.584625 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:54.718161 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349a2460-332d-4797-904f-bf75ddf7da40 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:54.720550 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81f2e6a0-6877-4612-966b-f522f52463c1 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:54.724777 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:54.763686 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5148d7a3-7e01-4a19-93d1-a8570a0079e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:54.764886 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5148d7a3-7e01-4a19-93d1-a8570a0079e1 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:54.765272 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5148d7a3-7e01-4a19-93d1-a8570a0079e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:54.765690 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5148d7a3-7e01-4a19-93d1-a8570a0079e1 None admin] PATH_INFO: `/v3/projects/064fc3c7af494c988e44e426f21b1c78` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:54.767214 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5148d7a3-7e01-4a19-93d1-a8570a0079e1 None admin] RBAC: Authorizing `identity:get_project(project_id=064fc3c7af494c988e44e426f21b1c78)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:54:54.767627 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f488d18d-18da-44b9-b785-279a5c5ad2b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:54.768554 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f488d18d-18da-44b9-b785-279a5c5ad2b5 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:54.768891 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f488d18d-18da-44b9-b785-279a5c5ad2b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:54.768990 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5148d7a3-7e01-4a19-93d1-a8570a0079e1 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:54:54.769166 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f488d18d-18da-44b9-b785-279a5c5ad2b5 None admin] PATH_INFO: `/v3/groups/4c84bf7a4c4f41da83d400606d236080` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:54.770387 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f488d18d-18da-44b9-b785-279a5c5ad2b5 None admin] RBAC: Authorizing `identity:get_group(group_id=4c84bf7a4c4f41da83d400606d236080)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:54:54.770978 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 207/411] 10.222.0.46 () {56 vars in 1186 bytes} [Thu May 26 14:54:54 2022] GET /identity/v3/projects/064fc3c7af494c988e44e426f21b1c78 => generated 344 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:54:54.771858 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f488d18d-18da-44b9-b785-279a5c5ad2b5 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:54:54.773731 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 205/412] 10.222.0.46 () {56 vars in 1180 bytes} [Thu May 26 14:54:54 2022] GET /identity/v3/groups/4c84bf7a4c4f41da83d400606d236080 => generated 226 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:54:54.776660 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62562612-8623-4a2b-99ed-f07fda369939 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:54.778331 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62562612-8623-4a2b-99ed-f07fda369939 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:54.778331 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62562612-8623-4a2b-99ed-f07fda369939 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:54.779147 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 208/413] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 14:54:54 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 14:54:54.779562 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f488d18d-18da-44b9-b785-279a5c5ad2b5 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:54.779996 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62562612-8623-4a2b-99ed-f07fda369939 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:54.894726 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:54.900102 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f488d18d-18da-44b9-b785-279a5c5ad2b5 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:54.906449 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62562612-8623-4a2b-99ed-f07fda369939 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:54.937359 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36c7ee0-0f2c-4be4-97c8-6e37829174d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:54.938695 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36c7ee0-0f2c-4be4-97c8-6e37829174d2 None admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:54.939165 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36c7ee0-0f2c-4be4-97c8-6e37829174d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:54.939716 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36c7ee0-0f2c-4be4-97c8-6e37829174d2 None admin] PATH_INFO: `/v3/projects/064fc3c7af494c988e44e426f21b1c78/groups/7757649a149e40c398b4898ac4f6d4a3/roles/a2e551773cbd4df084020676865b8222` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:54.942022 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36c7ee0-0f2c-4be4-97c8-6e37829174d2 None admin] RBAC: Authorizing `identity:create_grant(project_id=064fc3c7af494c988e44e426f21b1c78, group_id=7757649a149e40c398b4898ac4f6d4a3, role_id=a2e551773cbd4df084020676865b8222)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:54:54.942936 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7200fa1-638c-4677-af49-c6c72a0adb0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:54.943805 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7200fa1-638c-4677-af49-c6c72a0adb0c None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:54.944056 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36c7ee0-0f2c-4be4-97c8-6e37829174d2 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:54:54.944154 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7200fa1-638c-4677-af49-c6c72a0adb0c None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:54.944352 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7200fa1-638c-4677-af49-c6c72a0adb0c None admin] PATH_INFO: `/v3/projects/94ad8a7442da4d658f58d1543ae06493` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:54.945609 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7200fa1-638c-4677-af49-c6c72a0adb0c None admin] RBAC: Authorizing `identity:get_project(project_id=94ad8a7442da4d658f58d1543ae06493)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:54:54.947144 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7200fa1-638c-4677-af49-c6c72a0adb0c None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:54:54.949831 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 209/414] 10.222.0.46 () {56 vars in 1186 bytes} [Thu May 26 14:54:54 2022] GET /identity/v3/projects/94ad8a7442da4d658f58d1543ae06493 => generated 302 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:54:54.955498 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46946086-f8d3-439b-addc-a4da9e8f9263 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:54.955797 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46946086-f8d3-439b-addc-a4da9e8f9263 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:54.956070 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46946086-f8d3-439b-addc-a4da9e8f9263 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:54.962958 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 206/415] 10.222.0.46 () {58 vars in 1442 bytes} [Thu May 26 14:54:54 2022] PUT /identity/v3/projects/064fc3c7af494c988e44e426f21b1c78/groups/7757649a149e40c398b4898ac4f6d4a3/roles/a2e551773cbd4df084020676865b8222 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 14:54:54.969472 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36c7ee0-0f2c-4be4-97c8-6e37829174d2 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:54.985070 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-46946086-f8d3-439b-addc-a4da9e8f9263 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:54:54.986892 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-46946086-f8d3-439b-addc-a4da9e8f9263 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:54:55.007007 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:55.091136 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36c7ee0-0f2c-4be4-97c8-6e37829174d2 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:55.131205 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd6dbbc-1ca0-49e2-8226-14c7c79715cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:55.132508 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd6dbbc-1ca0-49e2-8226-14c7c79715cd None admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:55.133020 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd6dbbc-1ca0-49e2-8226-14c7c79715cd None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:55.133535 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd6dbbc-1ca0-49e2-8226-14c7c79715cd None admin] PATH_INFO: `/v3/projects/94ad8a7442da4d658f58d1543ae06493/groups/4c84bf7a4c4f41da83d400606d236080/roles/160115b2143c4b389d39cd93f43e1d01` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:55.136846 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd6dbbc-1ca0-49e2-8226-14c7c79715cd None admin] RBAC: Authorizing `identity:create_grant(project_id=94ad8a7442da4d658f58d1543ae06493, group_id=4c84bf7a4c4f41da83d400606d236080, role_id=160115b2143c4b389d39cd93f43e1d01)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:54:55.138658 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd6dbbc-1ca0-49e2-8226-14c7c79715cd None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:54:55.162857 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 207/416] 10.222.0.46 () {58 vars in 1442 bytes} [Thu May 26 14:54:54 2022] PUT /identity/v3/projects/94ad8a7442da4d658f58d1543ae06493/groups/4c84bf7a4c4f41da83d400606d236080/roles/160115b2143c4b389d39cd93f43e1d01 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 14:54:55.197349 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 210/417] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 14:54:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 14:54:55.234404 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd6dbbc-1ca0-49e2-8226-14c7c79715cd None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:55.346366 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:55.359167 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd6dbbc-1ca0-49e2-8226-14c7c79715cd None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:55.400979 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87819b55-933c-4a34-888c-7b85ab5f2807 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:55.401971 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87819b55-933c-4a34-888c-7b85ab5f2807 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:55.402258 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87819b55-933c-4a34-888c-7b85ab5f2807 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:55.402531 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87819b55-933c-4a34-888c-7b85ab5f2807 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:55.414815 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87819b55-933c-4a34-888c-7b85ab5f2807 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 14:54:55.423320 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-87819b55-933c-4a34-888c-7b85ab5f2807 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 26 14:54:55.423320 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:54:55.423320 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:54:55.423320 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:54:55.423320 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:54:55.423320 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:54:55.423320 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:54:55.423320 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:54:55.423320 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:54:55.423320 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:54:55.423320 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:54:55.423320 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:54:55.423320 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 14:54:55.423320 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 14:54:55.423320 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 14:54:55.423320 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 14:54:55.424270 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 14:54:55.424270 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 14:54:55.424270 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 14:54:55.424270 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 14:54:55.424270 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:54:55.424270 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:54:55.424270 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:54:55.424270 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:54:55.424270 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:54:55.424270 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:54:55.424270 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:54:55.424270 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 26 14:54:55.424270 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:54:55.424270 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 26 14:54:55.424270 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:54:55.424270 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:54:55.425053 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:54:55.425053 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 26 14:54:55.425053 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:54:55.425053 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 26 14:54:55.425053 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 14:54:55.425053 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 14:54:55.425053 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 14:54:55.425053 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 14:54:55.425053 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 14:54:55.425053 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 14:54:55.425053 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 26 14:54:55.425053 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 26 14:54:55.425053 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 208/418] 10.222.0.46 () {56 vars in 1099 bytes} [Thu May 26 14:54:55 2022] GET /identity/v3/roles/member => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:54:55.437935 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46946086-f8d3-439b-addc-a4da9e8f9263 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:55.601832 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:55.652953 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46946086-f8d3-439b-addc-a4da9e8f9263 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:55.716153 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf365aba-b595-4389-8d5f-bf2c0735fc73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:55.717221 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf365aba-b595-4389-8d5f-bf2c0735fc73 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:55.717725 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf365aba-b595-4389-8d5f-bf2c0735fc73 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:55.718199 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf365aba-b595-4389-8d5f-bf2c0735fc73 None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:55.719169 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf365aba-b595-4389-8d5f-bf2c0735fc73 None admin] RBAC: Adding query filter params (name=member) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:54:55.719668 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf365aba-b595-4389-8d5f-bf2c0735fc73 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:54:55.721241 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf365aba-b595-4389-8d5f-bf2c0735fc73 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:54:55.734261 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 211/419] 10.222.0.46 () {56 vars in 1101 bytes} [Thu May 26 14:54:55 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:54:55.741162 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87819b55-933c-4a34-888c-7b85ab5f2807 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:55.912254 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87819b55-933c-4a34-888c-7b85ab5f2807 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:55.952953 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabb502f-4a43-4b3d-9624-0bd5d8f9ce90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:55.954082 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabb502f-4a43-4b3d-9624-0bd5d8f9ce90 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:55.954565 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabb502f-4a43-4b3d-9624-0bd5d8f9ce90 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:55.955083 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabb502f-4a43-4b3d-9624-0bd5d8f9ce90 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:55.985930 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:56.072767 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68651995-31d9-4a23-b812-ed640c5acd9c None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:56.073193 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68651995-31d9-4a23-b812-ed640c5acd9c None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:56.073365 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68651995-31d9-4a23-b812-ed640c5acd9c None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:56.075004 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 212/420] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 14:54:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 14:54:56.077360 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabb502f-4a43-4b3d-9624-0bd5d8f9ce90 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:54:56.079928 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabb502f-4a43-4b3d-9624-0bd5d8f9ce90 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:54:56.080262 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2832dc87-b6b8-4746-a73b-0de28201a31a None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:56.080554 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2832dc87-b6b8-4746-a73b-0de28201a31a None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:56.080835 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2832dc87-b6b8-4746-a73b-0de28201a31a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:56.130617 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-2832dc87-b6b8-4746-a73b-0de28201a31a None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:54:56.132515 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-2832dc87-b6b8-4746-a73b-0de28201a31a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:54:56.311430 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 213/421] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 14:54:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 14:54:56.344837 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 209/422] 10.222.0.46 () {58 vars in 1282 bytes} [Thu May 26 14:54:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 604 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 14:54:56.350027 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2832dc87-b6b8-4746-a73b-0de28201a31a None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:56.365392 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabb502f-4a43-4b3d-9624-0bd5d8f9ce90 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:56.490814 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:56.543169 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabb502f-4a43-4b3d-9624-0bd5d8f9ce90 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:56.585605 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a1b6d7-5220-41b7-ba7e-a23a0cfd064e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:56.585939 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:56.587253 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a1b6d7-5220-41b7-ba7e-a23a0cfd064e None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:56.587850 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a1b6d7-5220-41b7-ba7e-a23a0cfd064e None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:56.588340 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a1b6d7-5220-41b7-ba7e-a23a0cfd064e None admin] PATH_INFO: `/v3/projects/e474420e5c124a3f86383fce1ef4c4fb` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:56.589976 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a1b6d7-5220-41b7-ba7e-a23a0cfd064e None admin] RBAC: Authorizing `identity:get_project(project_id=e474420e5c124a3f86383fce1ef4c4fb)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:54:56.591811 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a1b6d7-5220-41b7-ba7e-a23a0cfd064e None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:54:56.593743 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 210/423] 10.222.0.46 () {56 vars in 1186 bytes} [Thu May 26 14:54:56 2022] GET /identity/v3/projects/e474420e5c124a3f86383fce1ef4c4fb => generated 298 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:54:56.601528 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a1b6d7-5220-41b7-ba7e-a23a0cfd064e None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:56.628624 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2832dc87-b6b8-4746-a73b-0de28201a31a None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:56.708973 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70d4bcf-5e34-4f75-abd8-f97256ac91b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:56.710798 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70d4bcf-5e34-4f75-abd8-f97256ac91b3 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:56.711781 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70d4bcf-5e34-4f75-abd8-f97256ac91b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:56.712288 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70d4bcf-5e34-4f75-abd8-f97256ac91b3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:56.736712 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70d4bcf-5e34-4f75-abd8-f97256ac91b3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 14:54:56.756142 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-b70d4bcf-5e34-4f75-abd8-f97256ac91b3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 14:54:56.756142 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:54:56.756142 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:54:56.756142 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:54:56.756142 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:54:56.756142 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:54:56.756142 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:54:56.756142 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:54:56.756142 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:54:56.756142 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:54:56.756142 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:54:56.756142 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:54:56.756142 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 14:54:56.756142 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 14:54:56.756142 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 14:54:56.756142 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 14:54:56.757102 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 14:54:56.757102 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 14:54:56.757102 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 14:54:56.757102 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 14:54:56.757102 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:54:56.757102 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:54:56.757102 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:54:56.757102 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:54:56.757102 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:54:56.757102 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:54:56.757102 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:54:56.757102 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 26 14:54:56.757102 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:54:56.757102 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 26 14:54:56.757102 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:54:56.757102 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:54:56.757923 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:54:56.757923 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 26 14:54:56.757923 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:54:56.757923 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 26 14:54:56.757923 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 14:54:56.757923 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 14:54:56.757923 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 14:54:56.757923 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 14:54:56.757923 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 14:54:56.757923 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 14:54:56.757923 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 14:54:56.757923 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 26 14:54:56.759795 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 214/424] 10.222.0.46 () {56 vars in 1114 bytes} [Thu May 26 14:54:56 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 411 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:54:56.766300 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70d4bcf-5e34-4f75-abd8-f97256ac91b3 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:56.805853 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a1b6d7-5220-41b7-ba7e-a23a0cfd064e None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:56.845968 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:56.874540 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b073d6fa-fb74-4493-a702-8a5ca70de3d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:56.875770 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b073d6fa-fb74-4493-a702-8a5ca70de3d5 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:56.877401 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b073d6fa-fb74-4493-a702-8a5ca70de3d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:56.878095 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b073d6fa-fb74-4493-a702-8a5ca70de3d5 None admin] PATH_INFO: `/v3/groups/4c84bf7a4c4f41da83d400606d236080` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:56.881145 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b073d6fa-fb74-4493-a702-8a5ca70de3d5 None admin] RBAC: Authorizing `identity:get_group(group_id=4c84bf7a4c4f41da83d400606d236080)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:54:56.882850 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b073d6fa-fb74-4493-a702-8a5ca70de3d5 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:54:56.887595 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 211/425] 10.222.0.46 () {56 vars in 1180 bytes} [Thu May 26 14:54:56 2022] GET /identity/v3/groups/4c84bf7a4c4f41da83d400606d236080 => generated 226 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:54:56.893579 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b073d6fa-fb74-4493-a702-8a5ca70de3d5 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:56.974371 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70d4bcf-5e34-4f75-abd8-f97256ac91b3 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:57.046497 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501102c6-50ea-4613-8437-d29a0e7244f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:57.047528 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501102c6-50ea-4613-8437-d29a0e7244f9 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:57.048009 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501102c6-50ea-4613-8437-d29a0e7244f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:57.049139 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501102c6-50ea-4613-8437-d29a0e7244f9 None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:57.050134 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501102c6-50ea-4613-8437-d29a0e7244f9 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:54:57.050710 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501102c6-50ea-4613-8437-d29a0e7244f9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:54:57.052436 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501102c6-50ea-4613-8437-d29a0e7244f9 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:54:57.069602 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 215/426] 10.222.0.46 () {56 vars in 1111 bytes} [Thu May 26 14:54:56 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:54:57.078640 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501102c6-50ea-4613-8437-d29a0e7244f9 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:57.083562 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:57.104850 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b073d6fa-fb74-4493-a702-8a5ca70de3d5 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:57.192795 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501102c6-50ea-4613-8437-d29a0e7244f9 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:57.210738 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4cee67f-9faf-4ec1-915d-9e3a8b694a12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:57.211705 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cee67f-9faf-4ec1-915d-9e3a8b694a12 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:57.212002 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cee67f-9faf-4ec1-915d-9e3a8b694a12 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:57.212289 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4cee67f-9faf-4ec1-915d-9e3a8b694a12 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:57.213153 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cee67f-9faf-4ec1-915d-9e3a8b694a12 None admin] RBAC: Adding query filter params (group.id=4c84bf7a4c4f41da83d400606d236080, role.id=d4088d1845a24fbd9bd8975ce0b1d3d0, scope.project.id=e474420e5c124a3f86383fce1ef4c4fb) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:54:57.213696 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cee67f-9faf-4ec1-915d-9e3a8b694a12 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:54:57.215396 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4cee67f-9faf-4ec1-915d-9e3a8b694a12 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:54:57.225831 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 212/427] 10.222.0.46 () {56 vars in 1376 bytes} [Thu May 26 14:54:56 2022] GET /identity/v3/role_assignments?group.id=4c84bf7a4c4f41da83d400606d236080&scope.project.id=e474420e5c124a3f86383fce1ef4c4fb&role.id=d4088d1845a24fbd9bd8975ce0b1d3d0 => generated 611 bytes in 332 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:54:57.251047 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fbd9e6-d690-403f-a491-107fbdc4be53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:57.251904 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fbd9e6-d690-403f-a491-107fbdc4be53 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:57.252292 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fbd9e6-d690-403f-a491-107fbdc4be53 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:57.252673 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13fbd9e6-d690-403f-a491-107fbdc4be53 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:57.384468 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13fbd9e6-d690-403f-a491-107fbdc4be53 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:54:57.385931 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13fbd9e6-d690-403f-a491-107fbdc4be53 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:54:57.387972 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:57.443525 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfd4969-4f28-4bd1-8a5f-c52af717afa1 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:57.444596 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfd4969-4f28-4bd1-8a5f-c52af717afa1 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:57.445121 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfd4969-4f28-4bd1-8a5f-c52af717afa1 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:57.447210 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 213/428] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 14:54:57 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 14:54:57.453406 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ca6475-4815-4c4e-97ba-896440b32a2d None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:57.454100 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ca6475-4815-4c4e-97ba-896440b32a2d None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:57.454617 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ca6475-4815-4c4e-97ba-896440b32a2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:57.491713 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-34ca6475-4815-4c4e-97ba-896440b32a2d None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:54:57.494425 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-34ca6475-4815-4c4e-97ba-896440b32a2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:54:57.542545 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 216/429] 10.222.0.46 () {58 vars in 1282 bytes} [Thu May 26 14:54:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 464 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 14:54:57.551804 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fbd9e6-d690-403f-a491-107fbdc4be53 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:57.645968 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:57.687486 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 214/430] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 14:54:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 14:54:57.719182 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ca6475-4815-4c4e-97ba-896440b32a2d None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:57.742364 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13fbd9e6-d690-403f-a491-107fbdc4be53 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:57.746163 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:57.795230 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2073023d-57a8-4c44-99b9-aa3e182c8b74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:57.796705 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2073023d-57a8-4c44-99b9-aa3e182c8b74 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:57.797325 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2073023d-57a8-4c44-99b9-aa3e182c8b74 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:57.797802 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2073023d-57a8-4c44-99b9-aa3e182c8b74 None admin] PATH_INFO: `/v3/projects/e474420e5c124a3f86383fce1ef4c4fb` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:57.799441 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2073023d-57a8-4c44-99b9-aa3e182c8b74 None admin] RBAC: Authorizing `identity:get_project(project_id=e474420e5c124a3f86383fce1ef4c4fb)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:54:57.801361 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2073023d-57a8-4c44-99b9-aa3e182c8b74 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:54:57.804620 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 217/431] 10.222.0.46 () {56 vars in 1186 bytes} [Thu May 26 14:54:57 2022] GET /identity/v3/projects/e474420e5c124a3f86383fce1ef4c4fb => generated 298 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:54:57.810052 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2073023d-57a8-4c44-99b9-aa3e182c8b74 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:57.872335 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ca6475-4815-4c4e-97ba-896440b32a2d None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:57.923251 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955a85ff-84fe-4e13-8e31-cb8f1e7d73ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:57.924564 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955a85ff-84fe-4e13-8e31-cb8f1e7d73ed None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:57.925187 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955a85ff-84fe-4e13-8e31-cb8f1e7d73ed None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:57.926595 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-955a85ff-84fe-4e13-8e31-cb8f1e7d73ed None admin] PATH_INFO: `/v3/roles/member` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:57.940296 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2073023d-57a8-4c44-99b9-aa3e182c8b74 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:57.941900 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-955a85ff-84fe-4e13-8e31-cb8f1e7d73ed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 14:54:57.956694 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-955a85ff-84fe-4e13-8e31-cb8f1e7d73ed None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 26 14:54:57.956694 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:54:57.956694 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:54:57.956694 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:54:57.956694 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:54:57.956694 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:54:57.956694 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:54:57.956694 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:54:57.956694 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:54:57.956694 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:54:57.956694 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:54:57.956694 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:54:57.956694 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 14:54:57.956694 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 14:54:57.956694 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 14:54:57.956694 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 14:54:57.957827 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 14:54:57.957827 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 14:54:57.957827 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 14:54:57.957827 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 14:54:57.957827 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:54:57.957827 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:54:57.957827 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:54:57.957827 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:54:57.957827 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:54:57.957827 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:54:57.957827 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:54:57.957827 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 26 14:54:57.957827 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:54:57.957827 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 26 14:54:57.957827 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:54:57.957827 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:54:57.958621 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:54:57.958621 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 26 14:54:57.958621 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:54:57.958621 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 26 14:54:57.958621 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 14:54:57.958621 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 14:54:57.958621 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 14:54:57.958621 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 14:54:57.958621 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 14:54:57.958621 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 14:54:57.958621 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 26 14:54:57.958621 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 26 14:54:57.958621 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 215/432] 10.222.0.46 () {56 vars in 1099 bytes} [Thu May 26 14:54:57 2022] GET /identity/v3/roles/member => generated 84 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 14:54:57.965024 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955a85ff-84fe-4e13-8e31-cb8f1e7d73ed None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:57.987026 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69e6218-0db3-4e01-8848-76f8db3362db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:57.988058 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69e6218-0db3-4e01-8848-76f8db3362db None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:57.988430 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69e6218-0db3-4e01-8848-76f8db3362db None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:57.988767 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69e6218-0db3-4e01-8848-76f8db3362db None admin] PATH_INFO: `/v3/groups/4c84bf7a4c4f41da83d400606d236080` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:57.991458 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69e6218-0db3-4e01-8848-76f8db3362db None admin] RBAC: Authorizing `identity:get_group(group_id=4c84bf7a4c4f41da83d400606d236080)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:54:57.992972 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69e6218-0db3-4e01-8848-76f8db3362db None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:54:57.995500 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 218/433] 10.222.0.46 () {56 vars in 1180 bytes} [Thu May 26 14:54:57 2022] GET /identity/v3/groups/4c84bf7a4c4f41da83d400606d236080 => generated 226 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:54:57.996713 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47aaaf35-cf29-4df7-afc5-c3a8602d73a5 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:57.996955 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47aaaf35-cf29-4df7-afc5-c3a8602d73a5 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:57.997234 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47aaaf35-cf29-4df7-afc5-c3a8602d73a5 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:57.999063 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 219/434] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 14:54:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 14:54:58.004468 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8e603c-9202-4010-80ae-982f1eeef9f8 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:58.004779 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8e603c-9202-4010-80ae-982f1eeef9f8 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:58.005024 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8e603c-9202-4010-80ae-982f1eeef9f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:58.026314 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-3b8e603c-9202-4010-80ae-982f1eeef9f8 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:54:58.027952 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-3b8e603c-9202-4010-80ae-982f1eeef9f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:54:58.035801 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:58.073831 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:58.108556 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-955a85ff-84fe-4e13-8e31-cb8f1e7d73ed None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:58.140722 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 220/435] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 14:54:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 14:54:58.142479 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8e603c-9202-4010-80ae-982f1eeef9f8 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:58.154573 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360a7127-09f2-4402-b436-13de63205db2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:58.155251 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360a7127-09f2-4402-b436-13de63205db2 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:58.155588 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360a7127-09f2-4402-b436-13de63205db2 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:58.155966 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360a7127-09f2-4402-b436-13de63205db2 None admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:58.156818 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360a7127-09f2-4402-b436-13de63205db2 None admin] RBAC: Adding query filter params (name=member) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:54:58.157142 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360a7127-09f2-4402-b436-13de63205db2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:54:58.158768 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360a7127-09f2-4402-b436-13de63205db2 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:54:58.168506 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 216/436] 10.222.0.46 () {56 vars in 1101 bytes} [Thu May 26 14:54:57 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:54:58.169353 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360a7127-09f2-4402-b436-13de63205db2 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:58.253912 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8e603c-9202-4010-80ae-982f1eeef9f8 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:58.278950 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360a7127-09f2-4402-b436-13de63205db2 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:58.281207 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:58.288602 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712732db-fe91-4606-b304-df002d044a0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:58.289371 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712732db-fe91-4606-b304-df002d044a0e None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:58.289747 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712732db-fe91-4606-b304-df002d044a0e None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:58.290132 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712732db-fe91-4606-b304-df002d044a0e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:58.290958 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712732db-fe91-4606-b304-df002d044a0e None admin] RBAC: Adding query filter params (group.id=4c84bf7a4c4f41da83d400606d236080, role.id=160115b2143c4b389d39cd93f43e1d01, scope.project.id=e474420e5c124a3f86383fce1ef4c4fb) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:54:58.291268 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712732db-fe91-4606-b304-df002d044a0e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:54:58.292737 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712732db-fe91-4606-b304-df002d044a0e None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:54:58.298595 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 221/437] 10.222.0.46 () {56 vars in 1376 bytes} [Thu May 26 14:54:58 2022] GET /identity/v3/role_assignments?group.id=4c84bf7a4c4f41da83d400606d236080&scope.project.id=e474420e5c124a3f86383fce1ef4c4fb&role.id=160115b2143c4b389d39cd93f43e1d01 => generated 611 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:54:58.299374 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712732db-fe91-4606-b304-df002d044a0e None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:58.316955 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189e7af1-7277-41ea-95b6-9eeee9327ed8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:58.317839 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189e7af1-7277-41ea-95b6-9eeee9327ed8 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:58.318106 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189e7af1-7277-41ea-95b6-9eeee9327ed8 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:58.318879 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189e7af1-7277-41ea-95b6-9eeee9327ed8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:58.331602 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189e7af1-7277-41ea-95b6-9eeee9327ed8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 14:54:58.340422 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-189e7af1-7277-41ea-95b6-9eeee9327ed8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 14:54:58.340422 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:54:58.340422 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:54:58.340422 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:54:58.340422 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:54:58.340422 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:54:58.340422 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:54:58.340422 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:54:58.340422 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:54:58.340422 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:54:58.340422 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:54:58.340422 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:54:58.340422 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 14:54:58.340422 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 14:54:58.340422 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 14:54:58.340422 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 14:54:58.341401 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 14:54:58.341401 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 14:54:58.341401 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 14:54:58.341401 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 14:54:58.341401 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:54:58.341401 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:54:58.341401 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:54:58.341401 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:54:58.341401 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:54:58.341401 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:54:58.341401 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:54:58.341401 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 26 14:54:58.341401 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:54:58.341401 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 26 14:54:58.341401 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:54:58.341401 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:54:58.343105 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:54:58.343105 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 26 14:54:58.343105 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:54:58.343105 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 26 14:54:58.343105 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 14:54:58.343105 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 14:54:58.343105 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 14:54:58.343105 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 14:54:58.343105 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 14:54:58.343105 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 14:54:58.343105 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 14:54:58.343105 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 26 14:54:58.343105 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 217/438] 10.222.0.46 () {56 vars in 1096 bytes} [Thu May 26 14:54:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:54:58.343763 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2de3aae-cd5c-422b-9935-54e162eb981f None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:58.343763 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2de3aae-cd5c-422b-9935-54e162eb981f None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:58.343763 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2de3aae-cd5c-422b-9935-54e162eb981f None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:58.345232 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 218/439] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 14:54:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 14:54:58.350076 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:58.350319 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2de3aae-cd5c-422b-9935-54e162eb981f None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:58.430949 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712732db-fe91-4606-b304-df002d044a0e None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:58.468509 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2de3aae-cd5c-422b-9935-54e162eb981f None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:58.471514 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364d85e8-b1a1-46cb-b986-02fb736f9617 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:58.472173 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364d85e8-b1a1-46cb-b986-02fb736f9617 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:58.472451 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364d85e8-b1a1-46cb-b986-02fb736f9617 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:58.472719 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364d85e8-b1a1-46cb-b986-02fb736f9617 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:58.511611 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2019daea-7e93-45a2-a3bc-7823199f009e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:58.512207 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2019daea-7e93-45a2-a3bc-7823199f009e None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:58.512487 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2019daea-7e93-45a2-a3bc-7823199f009e None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:58.512760 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2019daea-7e93-45a2-a3bc-7823199f009e None admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:58.513534 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2019daea-7e93-45a2-a3bc-7823199f009e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:54:58.514144 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2019daea-7e93-45a2-a3bc-7823199f009e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:54:58.515198 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2019daea-7e93-45a2-a3bc-7823199f009e None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:54:58.524235 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 219/440] 10.222.0.46 () {56 vars in 1099 bytes} [Thu May 26 14:54:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 14:54:58.527151 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950ab33e-99f4-49de-9817-2dc5cdf709c1 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:58.527449 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950ab33e-99f4-49de-9817-2dc5cdf709c1 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:58.527727 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950ab33e-99f4-49de-9817-2dc5cdf709c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:58.548274 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-950ab33e-99f4-49de-9817-2dc5cdf709c1 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:54:58.549506 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-950ab33e-99f4-49de-9817-2dc5cdf709c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:54:58.558625 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364d85e8-b1a1-46cb-b986-02fb736f9617 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:54:58.560212 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364d85e8-b1a1-46cb-b986-02fb736f9617 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:54:58.572186 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:58.595287 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:58.627218 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 220/441] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 14:54:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 14:54:58.628457 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950ab33e-99f4-49de-9817-2dc5cdf709c1 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:58.686996 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 222/442] 10.222.0.46 () {58 vars in 1282 bytes} [Thu May 26 14:54:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 14:54:58.687854 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364d85e8-b1a1-46cb-b986-02fb736f9617 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:58.745739 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950ab33e-99f4-49de-9817-2dc5cdf709c1 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:58.779508 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0ad469-e6d1-4742-aea0-d6f160a104cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:58.780080 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0ad469-e6d1-4742-aea0-d6f160a104cb None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:58.780358 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0ad469-e6d1-4742-aea0-d6f160a104cb None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:58.780633 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0ad469-e6d1-4742-aea0-d6f160a104cb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:58.793188 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364d85e8-b1a1-46cb-b986-02fb736f9617 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:58.805192 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:58.822613 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:58.828087 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f7c677-4adc-4b41-b61f-e6df92580fa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:58.828897 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f7c677-4adc-4b41-b61f-e6df92580fa5 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:58.829181 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f7c677-4adc-4b41-b61f-e6df92580fa5 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:58.829475 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f7c677-4adc-4b41-b61f-e6df92580fa5 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:58.840045 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f7c677-4adc-4b41-b61f-e6df92580fa5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 14:54:58.848260 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-74f7c677-4adc-4b41-b61f-e6df92580fa5 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 14:54:58.848260 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:54:58.848260 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:54:58.848260 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:54:58.848260 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:54:58.848260 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:54:58.848260 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:54:58.848260 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:54:58.848260 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:54:58.848260 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:54:58.848260 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:54:58.848260 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:54:58.848260 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 14:54:58.848260 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 14:54:58.848260 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 14:54:58.848260 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 14:54:58.849216 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 14:54:58.849216 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 14:54:58.849216 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 14:54:58.849216 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 14:54:58.849216 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:54:58.849216 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:54:58.849216 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:54:58.849216 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:54:58.849216 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:54:58.849216 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:54:58.849216 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:54:58.849216 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 26 14:54:58.849216 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:54:58.849216 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 26 14:54:58.849216 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:54:58.849216 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:54:58.850217 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:54:58.850217 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 26 14:54:58.850217 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:54:58.850217 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 26 14:54:58.850217 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 14:54:58.850217 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 14:54:58.850217 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 14:54:58.850217 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 14:54:58.850217 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 14:54:58.850217 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 14:54:58.850217 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 14:54:58.850217 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 26 14:54:58.850992 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0ad469-e6d1-4742-aea0-d6f160a104cb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:54:58.851063 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 223/443] 10.222.0.46 () {56 vars in 1114 bytes} [Thu May 26 14:54:58 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:54:58.851137 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0ad469-e6d1-4742-aea0-d6f160a104cb None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:54:58.852238 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f7c677-4adc-4b41-b61f-e6df92580fa5 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:58.952149 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 221/444] 10.222.0.46 () {58 vars in 1282 bytes} [Thu May 26 14:54:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 14:54:58.952527 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f7c677-4adc-4b41-b61f-e6df92580fa5 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:58.952980 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0ad469-e6d1-4742-aea0-d6f160a104cb None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:58.988561 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3ebc8f-44d9-4a3b-92a4-09ed5f0a4523 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:58.989425 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3ebc8f-44d9-4a3b-92a4-09ed5f0a4523 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:58.989748 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3ebc8f-44d9-4a3b-92a4-09ed5f0a4523 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:58.990029 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3ebc8f-44d9-4a3b-92a4-09ed5f0a4523 None admin] PATH_INFO: `/v3/projects/94ad8a7442da4d658f58d1543ae06493` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:58.991224 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3ebc8f-44d9-4a3b-92a4-09ed5f0a4523 None admin] RBAC: Authorizing `identity:get_project(project_id=94ad8a7442da4d658f58d1543ae06493)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:54:58.993108 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3ebc8f-44d9-4a3b-92a4-09ed5f0a4523 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:54:58.995251 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 224/445] 10.222.0.46 () {56 vars in 1186 bytes} [Thu May 26 14:54:58 2022] GET /identity/v3/projects/94ad8a7442da4d658f58d1543ae06493 => generated 302 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:54:58.996336 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3ebc8f-44d9-4a3b-92a4-09ed5f0a4523 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:59.010834 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:59.054084 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0ad469-e6d1-4742-aea0-d6f160a104cb None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:59.058033 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:59.089532 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95a4b5a-35f0-4a00-806f-4712418c7acb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:59.090210 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95a4b5a-35f0-4a00-806f-4712418c7acb None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:59.090493 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95a4b5a-35f0-4a00-806f-4712418c7acb None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:59.090771 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95a4b5a-35f0-4a00-806f-4712418c7acb None admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:59.091538 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95a4b5a-35f0-4a00-806f-4712418c7acb None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:54:59.091846 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95a4b5a-35f0-4a00-806f-4712418c7acb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:54:59.093232 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95a4b5a-35f0-4a00-806f-4712418c7acb None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:54:59.101115 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 222/446] 10.222.0.46 () {56 vars in 1111 bytes} [Thu May 26 14:54:58 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:54:59.101734 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3ebc8f-44d9-4a3b-92a4-09ed5f0a4523 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:59.101930 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95a4b5a-35f0-4a00-806f-4712418c7acb None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:59.138819 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb918d6-1f65-48ee-b355-afb500a3ac02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:59.139677 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb918d6-1f65-48ee-b355-afb500a3ac02 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:59.139960 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb918d6-1f65-48ee-b355-afb500a3ac02 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:59.140237 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb918d6-1f65-48ee-b355-afb500a3ac02 None admin] PATH_INFO: `/v3/projects/064fc3c7af494c988e44e426f21b1c78` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:59.141408 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb918d6-1f65-48ee-b355-afb500a3ac02 None admin] RBAC: Authorizing `identity:get_project(project_id=064fc3c7af494c988e44e426f21b1c78)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:54:59.142871 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb918d6-1f65-48ee-b355-afb500a3ac02 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:54:59.144548 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 225/447] 10.222.0.46 () {56 vars in 1186 bytes} [Thu May 26 14:54:58 2022] GET /identity/v3/projects/064fc3c7af494c988e44e426f21b1c78 => generated 344 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:54:59.145308 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb918d6-1f65-48ee-b355-afb500a3ac02 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:59.204573 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95a4b5a-35f0-4a00-806f-4712418c7acb None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:59.221587 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:59.239377 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e795ad-6a08-4491-84ce-59143bdf8640 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:59.240244 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e795ad-6a08-4491-84ce-59143bdf8640 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:59.240530 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e795ad-6a08-4491-84ce-59143bdf8640 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:59.240800 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e795ad-6a08-4491-84ce-59143bdf8640 None admin] PATH_INFO: `/v3/groups/4c84bf7a4c4f41da83d400606d236080` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:59.242038 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e795ad-6a08-4491-84ce-59143bdf8640 None admin] RBAC: Authorizing `identity:get_group(group_id=4c84bf7a4c4f41da83d400606d236080)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:54:59.243051 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb918d6-1f65-48ee-b355-afb500a3ac02 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:59.243466 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e795ad-6a08-4491-84ce-59143bdf8640 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:54:59.245098 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 223/448] 10.222.0.46 () {56 vars in 1180 bytes} [Thu May 26 14:54:59 2022] GET /identity/v3/groups/4c84bf7a4c4f41da83d400606d236080 => generated 226 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:54:59.245885 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e795ad-6a08-4491-84ce-59143bdf8640 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:59.270371 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:59.279097 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a6d66d-9dc9-4dee-803f-20332bcb3160 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:59.279745 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a6d66d-9dc9-4dee-803f-20332bcb3160 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:59.280030 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a6d66d-9dc9-4dee-803f-20332bcb3160 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:59.280307 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a6d66d-9dc9-4dee-803f-20332bcb3160 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:59.344421 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e795ad-6a08-4491-84ce-59143bdf8640 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:59.346950 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a6d66d-9dc9-4dee-803f-20332bcb3160 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:54:59.348433 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a6d66d-9dc9-4dee-803f-20332bcb3160 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:54:59.379023 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7c7a51-a1eb-4ff8-ab7b-ab058dc5ce3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:59.379890 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7c7a51-a1eb-4ff8-ab7b-ab058dc5ce3a None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:59.380442 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7c7a51-a1eb-4ff8-ab7b-ab058dc5ce3a None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:59.380722 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7c7a51-a1eb-4ff8-ab7b-ab058dc5ce3a None admin] PATH_INFO: `/v3/groups/7757649a149e40c398b4898ac4f6d4a3` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:59.381931 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7c7a51-a1eb-4ff8-ab7b-ab058dc5ce3a None admin] RBAC: Authorizing `identity:get_group(group_id=7757649a149e40c398b4898ac4f6d4a3)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:54:59.383376 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7c7a51-a1eb-4ff8-ab7b-ab058dc5ce3a None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:54:59.384993 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 224/449] 10.222.0.46 () {56 vars in 1180 bytes} [Thu May 26 14:54:59 2022] GET /identity/v3/groups/7757649a149e40c398b4898ac4f6d4a3 => generated 229 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:54:59.386281 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7c7a51-a1eb-4ff8-ab7b-ab058dc5ce3a None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:59.443275 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:59.454026 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 226/450] 10.222.0.46 () {58 vars in 1282 bytes} [Thu May 26 14:54:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 14:54:59.454873 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a6d66d-9dc9-4dee-803f-20332bcb3160 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:59.485255 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:59.489584 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7c7a51-a1eb-4ff8-ab7b-ab058dc5ce3a None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:59.523755 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354aff0a-75c7-49bd-ba03-d644ee78cd10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:59.524408 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354aff0a-75c7-49bd-ba03-d644ee78cd10 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:59.524710 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354aff0a-75c7-49bd-ba03-d644ee78cd10 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:59.524985 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354aff0a-75c7-49bd-ba03-d644ee78cd10 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:59.525840 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354aff0a-75c7-49bd-ba03-d644ee78cd10 None admin] RBAC: Adding query filter params (group.id=4c84bf7a4c4f41da83d400606d236080, role.id=d4088d1845a24fbd9bd8975ce0b1d3d0, scope.project.id=94ad8a7442da4d658f58d1543ae06493) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:54:59.526145 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354aff0a-75c7-49bd-ba03-d644ee78cd10 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:54:59.527571 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354aff0a-75c7-49bd-ba03-d644ee78cd10 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:54:59.533667 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 225/451] 10.222.0.46 () {56 vars in 1376 bytes} [Thu May 26 14:54:59 2022] GET /identity/v3/role_assignments?group.id=4c84bf7a4c4f41da83d400606d236080&scope.project.id=94ad8a7442da4d658f58d1543ae06493&role.id=d4088d1845a24fbd9bd8975ce0b1d3d0 => generated 611 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:54:59.534449 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354aff0a-75c7-49bd-ba03-d644ee78cd10 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:59.568600 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a6d66d-9dc9-4dee-803f-20332bcb3160 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:59.613471 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36a0476-7403-47d0-9f03-61076c5fa48a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:59.614084 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36a0476-7403-47d0-9f03-61076c5fa48a None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:59.614378 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36a0476-7403-47d0-9f03-61076c5fa48a None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:59.614660 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36a0476-7403-47d0-9f03-61076c5fa48a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:59.615512 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36a0476-7403-47d0-9f03-61076c5fa48a None admin] RBAC: Adding query filter params (group.id=7757649a149e40c398b4898ac4f6d4a3, role.id=a2e551773cbd4df084020676865b8222, scope.project.id=064fc3c7af494c988e44e426f21b1c78) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:54:59.615830 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36a0476-7403-47d0-9f03-61076c5fa48a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:54:59.617400 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36a0476-7403-47d0-9f03-61076c5fa48a None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:54:59.629813 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 227/452] 10.222.0.46 () {56 vars in 1376 bytes} [Thu May 26 14:54:59 2022] GET /identity/v3/role_assignments?group.id=7757649a149e40c398b4898ac4f6d4a3&scope.project.id=064fc3c7af494c988e44e426f21b1c78&role.id=a2e551773cbd4df084020676865b8222 => generated 611 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:54:59.668473 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354aff0a-75c7-49bd-ba03-d644ee78cd10 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:59.691253 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:59.712511 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1910509-1b6e-4668-bd88-d74a04c0d28a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:59.714015 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1910509-1b6e-4668-bd88-d74a04c0d28a None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:59.714303 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1910509-1b6e-4668-bd88-d74a04c0d28a None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:59.714581 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1910509-1b6e-4668-bd88-d74a04c0d28a None admin] PATH_INFO: `/v3/projects/94ad8a7442da4d658f58d1543ae06493` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:59.715959 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1910509-1b6e-4668-bd88-d74a04c0d28a None admin] RBAC: Authorizing `identity:get_project(project_id=94ad8a7442da4d658f58d1543ae06493)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:54:59.717505 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1910509-1b6e-4668-bd88-d74a04c0d28a None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:54:59.719492 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 226/453] 10.222.0.46 () {56 vars in 1186 bytes} [Thu May 26 14:54:59 2022] GET /identity/v3/projects/94ad8a7442da4d658f58d1543ae06493 => generated 302 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:54:59.725238 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36a0476-7403-47d0-9f03-61076c5fa48a None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:54:59.853267 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:54:59.864544 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36a0476-7403-47d0-9f03-61076c5fa48a None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:54:59.903649 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85a7c9e-587c-4602-b355-61f75a129f72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:54:59.904510 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85a7c9e-587c-4602-b355-61f75a129f72 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:54:59.904796 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85a7c9e-587c-4602-b355-61f75a129f72 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:54:59.905063 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85a7c9e-587c-4602-b355-61f75a129f72 None admin] PATH_INFO: `/v3/groups/4c84bf7a4c4f41da83d400606d236080` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:54:59.906270 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85a7c9e-587c-4602-b355-61f75a129f72 None admin] RBAC: Authorizing `identity:get_group(group_id=4c84bf7a4c4f41da83d400606d236080)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:54:59.907697 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85a7c9e-587c-4602-b355-61f75a129f72 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:54:59.909373 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 228/454] 10.222.0.46 () {56 vars in 1180 bytes} [Thu May 26 14:54:59 2022] GET /identity/v3/groups/4c84bf7a4c4f41da83d400606d236080 => generated 226 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:54:59.915497 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1910509-1b6e-4668-bd88-d74a04c0d28a None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:00.022246 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1910509-1b6e-4668-bd88-d74a04c0d28a None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:00.058262 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fdcb825-4f33-4aaa-bf89-295ab118e2b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:00.058928 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdcb825-4f33-4aaa-bf89-295ab118e2b4 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:00.059216 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdcb825-4f33-4aaa-bf89-295ab118e2b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:00.059498 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fdcb825-4f33-4aaa-bf89-295ab118e2b4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:00.060329 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fdcb825-4f33-4aaa-bf89-295ab118e2b4 None admin] RBAC: Adding query filter params (group.id=4c84bf7a4c4f41da83d400606d236080, role.id=160115b2143c4b389d39cd93f43e1d01, scope.project.id=94ad8a7442da4d658f58d1543ae06493) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:55:00.060645 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fdcb825-4f33-4aaa-bf89-295ab118e2b4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:00.062152 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fdcb825-4f33-4aaa-bf89-295ab118e2b4 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:00.068952 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 227/455] 10.222.0.46 () {56 vars in 1376 bytes} [Thu May 26 14:54:59 2022] GET /identity/v3/role_assignments?group.id=4c84bf7a4c4f41da83d400606d236080&scope.project.id=94ad8a7442da4d658f58d1543ae06493&role.id=160115b2143c4b389d39cd93f43e1d01 => generated 611 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:55:03.118793 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3d8c90-ba5e-41b0-a6d6-182b977d5a27 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:03.119642 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3d8c90-ba5e-41b0-a6d6-182b977d5a27 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:03.120061 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3d8c90-ba5e-41b0-a6d6-182b977d5a27 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:03.121883 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 229/456] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 14:55:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 14:55:03.127707 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf3dfe9-a37c-4619-884e-8ff9c5a1e22c None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:03.128517 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf3dfe9-a37c-4619-884e-8ff9c5a1e22c None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:03.128941 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf3dfe9-a37c-4619-884e-8ff9c5a1e22c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:03.154459 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-6bf3dfe9-a37c-4619-884e-8ff9c5a1e22c None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:55:03.156297 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-6bf3dfe9-a37c-4619-884e-8ff9c5a1e22c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:55:03.211710 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e24e943-45d0-4112-8c2b-d197b6eb0da7 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:03.212302 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e24e943-45d0-4112-8c2b-d197b6eb0da7 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:03.212740 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e24e943-45d0-4112-8c2b-d197b6eb0da7 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:03.215432 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 230/457] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 14:55:03 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 14:55:03.220843 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4017be-273c-4a68-867c-dfe8181454c5 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:03.221373 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4017be-273c-4a68-867c-dfe8181454c5 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:03.221857 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4017be-273c-4a68-867c-dfe8181454c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:03.247747 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-bf4017be-273c-4a68-867c-dfe8181454c5 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:55:03.251089 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-bf4017be-273c-4a68-867c-dfe8181454c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:55:03.253858 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:03.297646 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 228/458] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 14:55:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 14:55:03.342209 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf3dfe9-a37c-4619-884e-8ff9c5a1e22c None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:03.372414 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 231/459] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 14:55:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 14:55:03.397379 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4017be-273c-4a68-867c-dfe8181454c5 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:03.494175 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf3dfe9-a37c-4619-884e-8ff9c5a1e22c None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:03.500064 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:03.512273 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4017be-273c-4a68-867c-dfe8181454c5 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:03.534116 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27891130-8e4a-4273-be0e-fa19baa48c58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:03.535220 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27891130-8e4a-4273-be0e-fa19baa48c58 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:03.535345 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27891130-8e4a-4273-be0e-fa19baa48c58 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:03.535576 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27891130-8e4a-4273-be0e-fa19baa48c58 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:03.547784 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27891130-8e4a-4273-be0e-fa19baa48c58 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:03.549565 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27891130-8e4a-4273-be0e-fa19baa48c58 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:03.554535 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-266a1e26-b649-48c2-865e-348d70f206d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:03.555584 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266a1e26-b649-48c2-865e-348d70f206d3 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:03.556072 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266a1e26-b649-48c2-865e-348d70f206d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:03.556547 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-266a1e26-b649-48c2-865e-348d70f206d3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:03.568048 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266a1e26-b649-48c2-865e-348d70f206d3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:03.568491 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-27891130-8e4a-4273-be0e-fa19baa48c58 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 14:55:03.568491 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:55:03.568491 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:55:03.568491 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:55:03.568491 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:55:03.568491 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:55:03.568491 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:55:03.568491 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:55:03.568491 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:55:03.568491 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:55:03.568491 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:55:03.568491 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:55:03.568491 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 14:55:03.568491 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 14:55:03.568491 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 14:55:03.569334 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 14:55:03.569334 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:55:03.569334 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:55:03.569334 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:55:03.569334 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:55:03.569334 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:55:03.569334 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:55:03.569334 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:55:03.569334 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 26 14:55:03.569334 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:55:03.569334 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 26 14:55:03.569334 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:55:03.569334 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:55:03.569334 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:55:03.569334 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 26 14:55:03.569334 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:55:03.569334 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 26 14:55:03.570219 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 14:55:03.570219 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 14:55:03.570219 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 14:55:03.570219 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 26 14:55:03.570219 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 229/460] 10.222.0.46 () {56 vars in 1108 bytes} [Thu May 26 14:55:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 14:55:03.571379 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd947c0a-b4cd-42d4-9f07-6073c367c876 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:03.571667 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd947c0a-b4cd-42d4-9f07-6073c367c876 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:03.571940 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd947c0a-b4cd-42d4-9f07-6073c367c876 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:03.572869 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-266a1e26-b649-48c2-865e-348d70f206d3 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:03.577331 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 230/461] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 14:55:03 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 14:55:03.578200 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd947c0a-b4cd-42d4-9f07-6073c367c876 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:03.593744 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-266a1e26-b649-48c2-865e-348d70f206d3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 14:55:03.593744 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:55:03.593744 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:55:03.593744 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:55:03.593744 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:55:03.593744 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:55:03.593744 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:55:03.593744 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:55:03.593744 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:55:03.593744 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:55:03.593744 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:55:03.593744 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:55:03.593744 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 14:55:03.593744 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 14:55:03.593744 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 14:55:03.594653 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 14:55:03.594653 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:55:03.594653 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:55:03.594653 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:55:03.594653 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:55:03.594653 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:55:03.594653 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:55:03.594653 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:55:03.594653 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 26 14:55:03.594653 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:55:03.594653 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 26 14:55:03.594653 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:55:03.594653 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:55:03.594653 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:55:03.594653 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 26 14:55:03.594653 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:55:03.594653 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 26 14:55:03.595591 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 14:55:03.595591 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 14:55:03.595591 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 14:55:03.595591 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 26 14:55:03.596952 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 232/462] 10.222.0.46 () {56 vars in 1108 bytes} [Thu May 26 14:55:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 14:55:03.598348 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c65c5e7-ff7d-4f42-9a6a-998a8886091f None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:03.598640 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c65c5e7-ff7d-4f42-9a6a-998a8886091f None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:03.598919 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c65c5e7-ff7d-4f42-9a6a-998a8886091f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:03.618060 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-5c65c5e7-ff7d-4f42-9a6a-998a8886091f None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:55:03.619478 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-5c65c5e7-ff7d-4f42-9a6a-998a8886091f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:55:03.629315 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:03.696679 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 233/463] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 14:55:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 14:55:03.698365 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647571a5-2dca-4f80-8493-006944583c36 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:03.698656 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647571a5-2dca-4f80-8493-006944583c36 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:03.698967 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647571a5-2dca-4f80-8493-006944583c36 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:03.704499 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 234/464] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 14:55:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 14:55:03.704499 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647571a5-2dca-4f80-8493-006944583c36 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:03.725712 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd947c0a-b4cd-42d4-9f07-6073c367c876 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:03.765792 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9840be8-9a27-4a47-94ba-1a4efb62d7e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:03.766464 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9840be8-9a27-4a47-94ba-1a4efb62d7e4 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:03.766761 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9840be8-9a27-4a47-94ba-1a4efb62d7e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:03.767028 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9840be8-9a27-4a47-94ba-1a4efb62d7e4 None admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:03.767750 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9840be8-9a27-4a47-94ba-1a4efb62d7e4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:55:03.768057 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9840be8-9a27-4a47-94ba-1a4efb62d7e4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:03.769425 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9840be8-9a27-4a47-94ba-1a4efb62d7e4 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:03.782538 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 231/465] 10.222.0.46 () {56 vars in 1109 bytes} [Thu May 26 14:55:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:55:03.783381 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883492da-25ff-4746-a1f2-ecaab05b1b68 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:03.783661 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883492da-25ff-4746-a1f2-ecaab05b1b68 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:03.783936 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883492da-25ff-4746-a1f2-ecaab05b1b68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:03.790207 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:03.800522 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-883492da-25ff-4746-a1f2-ecaab05b1b68 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:55:03.801726 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-883492da-25ff-4746-a1f2-ecaab05b1b68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:55:03.826070 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647571a5-2dca-4f80-8493-006944583c36 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:03.863316 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27dd38f2-db41-452d-8428-927f03a82685 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:03.863984 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dd38f2-db41-452d-8428-927f03a82685 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:03.864266 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dd38f2-db41-452d-8428-927f03a82685 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:03.864545 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dd38f2-db41-452d-8428-927f03a82685 None admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:03.865267 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27dd38f2-db41-452d-8428-927f03a82685 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:55:03.865614 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27dd38f2-db41-452d-8428-927f03a82685 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:03.867122 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27dd38f2-db41-452d-8428-927f03a82685 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:03.880753 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 232/466] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 14:55:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 14:55:03.880753 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883492da-25ff-4746-a1f2-ecaab05b1b68 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:03.881021 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 235/467] 10.222.0.46 () {56 vars in 1109 bytes} [Thu May 26 14:55:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:55:03.885715 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27dd38f2-db41-452d-8428-927f03a82685 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:03.907273 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:03.996308 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883492da-25ff-4746-a1f2-ecaab05b1b68 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:03.997249 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27dd38f2-db41-452d-8428-927f03a82685 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:04.032935 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:04.037821 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bafdde-3769-439e-a30c-87414095f182 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:04.038632 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bafdde-3769-439e-a30c-87414095f182 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:04.038804 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43065ec-f556-40c5-9cda-40963442014a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:04.038936 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bafdde-3769-439e-a30c-87414095f182 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:04.039179 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9bafdde-3769-439e-a30c-87414095f182 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:04.039442 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43065ec-f556-40c5-9cda-40963442014a None admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:04.040134 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43065ec-f556-40c5-9cda-40963442014a None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:04.040412 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43065ec-f556-40c5-9cda-40963442014a None admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:04.041239 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43065ec-f556-40c5-9cda-40963442014a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:04.042876 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43065ec-f556-40c5-9cda-40963442014a None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:04.049244 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9bafdde-3769-439e-a30c-87414095f182 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:04.050730 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9bafdde-3769-439e-a30c-87414095f182 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:04.068600 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 236/468] 10.222.0.46 () {60 vars in 1132 bytes} [Thu May 26 14:55:03 2022] POST /identity/v3/users => generated 246 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 14:55:04.071223 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43065ec-f556-40c5-9cda-40963442014a None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:04.073133 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-f9bafdde-3769-439e-a30c-87414095f182 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 14:55:04.073133 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:55:04.073133 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:55:04.073133 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:55:04.073133 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:55:04.073133 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:55:04.073133 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:55:04.073133 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:55:04.073133 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:55:04.073133 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:55:04.073133 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:55:04.073133 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:55:04.073133 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 14:55:04.073133 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 14:55:04.073133 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 14:55:04.075001 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 14:55:04.075001 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:55:04.075001 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:55:04.075001 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:55:04.075001 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:55:04.075001 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:55:04.075001 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:55:04.075001 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:55:04.075001 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 26 14:55:04.075001 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:55:04.075001 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 26 14:55:04.075001 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:55:04.075001 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:55:04.075001 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:55:04.075001 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 26 14:55:04.075001 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:55:04.075001 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 26 14:55:04.076138 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 14:55:04.076138 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 14:55:04.076138 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 14:55:04.076138 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 26 14:55:04.080496 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 233/469] 10.222.0.46 () {56 vars in 1108 bytes} [Thu May 26 14:55:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:55:04.081398 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bafdde-3769-439e-a30c-87414095f182 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:04.179943 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:04.226360 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9bafdde-3769-439e-a30c-87414095f182 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:04.227046 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43065ec-f556-40c5-9cda-40963442014a None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:04.268037 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805df0c4-3f2f-4095-88b4-de95e1af4849 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:04.269068 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805df0c4-3f2f-4095-88b4-de95e1af4849 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:04.269582 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805df0c4-3f2f-4095-88b4-de95e1af4849 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:04.270042 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805df0c4-3f2f-4095-88b4-de95e1af4849 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:04.275893 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88316190-4b50-472e-88f8-fddebfacfdb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:04.276717 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88316190-4b50-472e-88f8-fddebfacfdb8 None admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:04.277038 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88316190-4b50-472e-88f8-fddebfacfdb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:04.277323 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88316190-4b50-472e-88f8-fddebfacfdb8 None admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:04.278144 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88316190-4b50-472e-88f8-fddebfacfdb8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:04.279714 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88316190-4b50-472e-88f8-fddebfacfdb8 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:04.286010 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805df0c4-3f2f-4095-88b4-de95e1af4849 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:04.287839 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805df0c4-3f2f-4095-88b4-de95e1af4849 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:04.303179 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 237/470] 10.222.0.46 () {60 vars in 1132 bytes} [Thu May 26 14:55:04 2022] POST /identity/v3/users => generated 248 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 14:55:04.304148 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88316190-4b50-472e-88f8-fddebfacfdb8 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:04.311887 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-805df0c4-3f2f-4095-88b4-de95e1af4849 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 14:55:04.311887 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:55:04.311887 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:55:04.311887 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:55:04.311887 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:55:04.311887 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:55:04.311887 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:55:04.311887 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:55:04.311887 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:55:04.311887 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:55:04.311887 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:55:04.311887 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:55:04.311887 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 14:55:04.311887 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 14:55:04.311887 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 14:55:04.312806 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 14:55:04.312806 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:55:04.312806 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:55:04.312806 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:55:04.312806 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:55:04.312806 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:55:04.312806 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:55:04.312806 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:55:04.312806 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 26 14:55:04.312806 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:55:04.312806 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 26 14:55:04.312806 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:55:04.312806 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:55:04.312806 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:55:04.312806 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 26 14:55:04.312806 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:55:04.312806 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 26 14:55:04.313698 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 14:55:04.313698 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 14:55:04.313698 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 14:55:04.313698 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 26 14:55:04.313698 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 234/471] 10.222.0.46 () {56 vars in 1108 bytes} [Thu May 26 14:55:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:55:04.319762 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805df0c4-3f2f-4095-88b4-de95e1af4849 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:04.369159 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:04.453401 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88316190-4b50-472e-88f8-fddebfacfdb8 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:04.479966 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805df0c4-3f2f-4095-88b4-de95e1af4849 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:04.497847 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3555c581-a036-4edb-a274-260d452e37f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:04.498533 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3555c581-a036-4edb-a274-260d452e37f4 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:04.498817 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3555c581-a036-4edb-a274-260d452e37f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:04.499089 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3555c581-a036-4edb-a274-260d452e37f4 None admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:04.499838 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3555c581-a036-4edb-a274-260d452e37f4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:55:04.500151 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3555c581-a036-4edb-a274-260d452e37f4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:04.501738 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3555c581-a036-4edb-a274-260d452e37f4 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:04.514217 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:04.515125 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 238/472] 10.222.0.46 () {56 vars in 1109 bytes} [Thu May 26 14:55:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:55:04.516364 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c559c814-f3d6-4327-bb90-035373798f90 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:04.516546 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c559c814-f3d6-4327-bb90-035373798f90 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:04.516823 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c559c814-f3d6-4327-bb90-035373798f90 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:04.518420 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 239/473] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 14:55:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 14:55:04.522300 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c559c814-f3d6-4327-bb90-035373798f90 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:04.527168 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82500950-0043-4119-a0cf-de893218d5c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:04.528140 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82500950-0043-4119-a0cf-de893218d5c3 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:04.528425 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82500950-0043-4119-a0cf-de893218d5c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:04.528702 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82500950-0043-4119-a0cf-de893218d5c3 None admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:04.529479 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82500950-0043-4119-a0cf-de893218d5c3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:55:04.530627 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82500950-0043-4119-a0cf-de893218d5c3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:04.532126 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82500950-0043-4119-a0cf-de893218d5c3 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:04.545764 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 235/474] 10.222.0.46 () {56 vars in 1109 bytes} [Thu May 26 14:55:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:55:04.547526 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4f9308-e265-4f05-b49b-b5daf33dfd4d None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:04.547526 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4f9308-e265-4f05-b49b-b5daf33dfd4d None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:04.547719 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4f9308-e265-4f05-b49b-b5daf33dfd4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:04.567021 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-7c4f9308-e265-4f05-b49b-b5daf33dfd4d None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:55:04.567645 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-7c4f9308-e265-4f05-b49b-b5daf33dfd4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:55:04.657626 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 236/475] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 14:55:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 14:55:04.658579 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4f9308-e265-4f05-b49b-b5daf33dfd4d None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:04.658688 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c559c814-f3d6-4327-bb90-035373798f90 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:04.661189 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:04.697928 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef9660b-9de3-4151-86d0-3cc4d4d92081 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:04.698611 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef9660b-9de3-4151-86d0-3cc4d4d92081 None admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:04.698988 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef9660b-9de3-4151-86d0-3cc4d4d92081 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:04.699269 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef9660b-9de3-4151-86d0-3cc4d4d92081 None admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:04.700032 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef9660b-9de3-4151-86d0-3cc4d4d92081 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:04.701501 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef9660b-9de3-4151-86d0-3cc4d4d92081 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:04.728291 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 240/476] 10.222.0.46 () {60 vars in 1132 bytes} [Thu May 26 14:55:04 2022] POST /identity/v3/users => generated 248 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 14:55:04.735158 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef9660b-9de3-4151-86d0-3cc4d4d92081 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:04.818347 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4f9308-e265-4f05-b49b-b5daf33dfd4d None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:04.853839 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:04.884964 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d72c9f-58fe-4f0a-b3b5-54c2004d0831 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:04.885758 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d72c9f-58fe-4f0a-b3b5-54c2004d0831 None admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:04.886047 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d72c9f-58fe-4f0a-b3b5-54c2004d0831 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:04.887644 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d72c9f-58fe-4f0a-b3b5-54c2004d0831 None admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:04.888411 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d72c9f-58fe-4f0a-b3b5-54c2004d0831 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:04.889962 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d72c9f-58fe-4f0a-b3b5-54c2004d0831 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:04.936535 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 237/477] 10.222.0.46 () {60 vars in 1132 bytes} [Thu May 26 14:55:04 2022] POST /identity/v3/users => generated 249 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 14:55:04.943493 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef9660b-9de3-4151-86d0-3cc4d4d92081 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:04.999717 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420a7c00-a04f-4be9-8aec-5f04a50f17ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:05.000333 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420a7c00-a04f-4be9-8aec-5f04a50f17ef None admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:05.000617 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420a7c00-a04f-4be9-8aec-5f04a50f17ef None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:05.000901 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420a7c00-a04f-4be9-8aec-5f04a50f17ef None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:05.001784 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420a7c00-a04f-4be9-8aec-5f04a50f17ef None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:05.004141 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420a7c00-a04f-4be9-8aec-5f04a50f17ef None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:05.013084 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.sql.core [None req-420a7c00-a04f-4be9-8aec-5f04a50f17ef None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 26 14:55:05.013084 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] May 26 14:55:05.013084 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [parameters: {'id': 'ed1fdadfe3364bef82a1cf8e8f98de26', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] May 26 14:55:05.013084 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80168) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 26 14:55:05.042524 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-420a7c00-a04f-4be9-8aec-5f04a50f17ef None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. May 26 14:55:05.042524 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:55:05.042524 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 26 14:55:05.042524 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application self.dialect.do_execute( May 26 14:55:05.042524 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 26 14:55:05.042524 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 26 14:55:05.042524 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 26 14:55:05.042524 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application result = self._query(query) May 26 14:55:05.042524 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 26 14:55:05.042524 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application conn.query(q) May 26 14:55:05.042524 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 26 14:55:05.042524 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 26 14:55:05.042524 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 26 14:55:05.042524 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application result.read() May 26 14:55:05.042524 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 26 14:55:05.043479 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 26 14:55:05.043479 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 26 14:55:05.043479 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application packet.raise_for_error() May 26 14:55:05.043479 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 26 14:55:05.043479 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 26 14:55:05.043479 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 26 14:55:05.043479 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 26 14:55:05.043479 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 26 14:55:05.043479 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 26 14:55:05.043479 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 26 14:55:05.043479 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 26 14:55:05.043479 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:55:05.043479 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 26 14:55:05.043479 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 26 14:55:05.043479 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role May 26 14:55:05.043479 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return ref.to_dict() May 26 14:55:05.043479 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 26 14:55:05.043479 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application next(self.gen) May 26 14:55:05.044895 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 26 14:55:05.044895 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application yield resource May 26 14:55:05.044895 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 26 14:55:05.044895 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application next(self.gen) May 26 14:55:05.044895 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 26 14:55:05.044895 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application self.session.rollback() May 26 14:55:05.044895 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 26 14:55:05.044895 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application self.force_reraise() May 26 14:55:05.044895 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 26 14:55:05.044895 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise self.value May 26 14:55:05.044895 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 26 14:55:05.044895 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 26 14:55:05.044895 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 26 14:55:05.044895 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application session.commit() May 26 14:55:05.044895 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 26 14:55:05.044895 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 26 14:55:05.044895 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 26 14:55:05.045809 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application self._prepare_impl() May 26 14:55:05.045809 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 26 14:55:05.045809 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application self.session.flush() May 26 14:55:05.045809 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 26 14:55:05.045809 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application self._flush(objects) May 26 14:55:05.045809 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 26 14:55:05.045809 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 26 14:55:05.045809 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 26 14:55:05.045809 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application compat.raise_( May 26 14:55:05.045809 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 26 14:55:05.045809 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception May 26 14:55:05.045809 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 26 14:55:05.045809 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application flush_context.execute() May 26 14:55:05.045809 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 26 14:55:05.045809 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rec.execute(self) May 26 14:55:05.045809 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 26 14:55:05.045809 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 26 14:55:05.046687 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 26 14:55:05.046687 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application _emit_insert_statements( May 26 14:55:05.046687 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements May 26 14:55:05.046687 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application c = connection._execute_20( May 26 14:55:05.046687 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 26 14:55:05.046687 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 26 14:55:05.046687 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 26 14:55:05.046687 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 26 14:55:05.046687 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 26 14:55:05.046687 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ret = self._execute_context( May 26 14:55:05.046687 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 26 14:55:05.046687 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 26 14:55:05.046687 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 26 14:55:05.046687 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 26 14:55:05.046687 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 26 14:55:05.046687 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception May 26 14:55:05.047492 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 26 14:55:05.047492 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application self.dialect.do_execute( May 26 14:55:05.047492 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 26 14:55:05.047492 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 26 14:55:05.047492 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 26 14:55:05.047492 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application result = self._query(query) May 26 14:55:05.047492 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 26 14:55:05.047492 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application conn.query(q) May 26 14:55:05.047492 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 26 14:55:05.047492 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 26 14:55:05.047492 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 26 14:55:05.047492 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application result.read() May 26 14:55:05.047492 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 26 14:55:05.047492 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 26 14:55:05.047492 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 26 14:55:05.047492 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application packet.raise_for_error() May 26 14:55:05.048275 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 26 14:55:05.048275 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 26 14:55:05.048275 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 26 14:55:05.048275 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 26 14:55:05.048275 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 26 14:55:05.048275 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] May 26 14:55:05.048275 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [parameters: {'id': 'ed1fdadfe3364bef82a1cf8e8f98de26', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] May 26 14:55:05.048275 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 26 14:55:05.048275 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 26 14:55:05.048275 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 14:55:05.048275 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 26 14:55:05.048275 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:55:05.048275 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:55:05.048275 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:55:05.048275 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:55:05.048275 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:55:05.049062 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:55:05.049062 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:55:05.049062 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:55:05.049062 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:55:05.049062 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:55:05.049062 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:55:05.049062 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post May 26 14:55:05.049062 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( May 26 14:55:05.049062 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:55:05.049062 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:55:05.049062 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role May 26 14:55:05.049062 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) May 26 14:55:05.049062 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 26 14:55:05.049062 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 26 14:55:05.049062 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. May 26 14:55:05.049062 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 26 14:55:05.050404 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 241/478] 10.222.0.46 () {60 vars in 1131 bytes} [Thu May 26 14:55:04 2022] POST /identity/v3/roles => generated 144 bytes in 316 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) May 26 14:55:05.052806 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d72c9f-58fe-4f0a-b3b5-54c2004d0831 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:05.174001 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d72c9f-58fe-4f0a-b3b5-54c2004d0831 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:05.211319 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc68f663-8a13-4954-869c-db532d0ed5cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:05.212789 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc68f663-8a13-4954-869c-db532d0ed5cc None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:05.213213 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc68f663-8a13-4954-869c-db532d0ed5cc None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:05.213653 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc68f663-8a13-4954-869c-db532d0ed5cc None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:05.226122 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc68f663-8a13-4954-869c-db532d0ed5cc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 14:55:05.234737 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-dc68f663-8a13-4954-869c-db532d0ed5cc None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 14:55:05.234737 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:55:05.234737 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:55:05.234737 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:55:05.234737 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:55:05.234737 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:55:05.234737 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:55:05.234737 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:55:05.234737 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:55:05.234737 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:55:05.234737 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:55:05.234737 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:55:05.234737 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 14:55:05.234737 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 14:55:05.234737 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 14:55:05.234737 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 14:55:05.235784 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 14:55:05.235784 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 14:55:05.235784 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 14:55:05.235784 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 14:55:05.235784 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:55:05.235784 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:55:05.235784 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:55:05.235784 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:55:05.235784 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:55:05.235784 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:55:05.235784 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:55:05.235784 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 26 14:55:05.235784 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:55:05.235784 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 26 14:55:05.235784 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:55:05.235784 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:55:05.236755 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:55:05.236755 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 26 14:55:05.236755 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:55:05.236755 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 26 14:55:05.236755 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 14:55:05.236755 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 14:55:05.236755 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 14:55:05.236755 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 14:55:05.236755 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 14:55:05.236755 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 14:55:05.236755 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 26 14:55:05.236755 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 26 14:55:05.239314 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 238/479] 10.222.0.46 () {56 vars in 1114 bytes} [Thu May 26 14:55:05 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:55:05.245045 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420a7c00-a04f-4be9-8aec-5f04a50f17ef None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:05.285626 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:05.366758 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420a7c00-a04f-4be9-8aec-5f04a50f17ef None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:05.403649 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796b3e89-b487-47cb-9931-31f838c5d2cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:05.404311 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796b3e89-b487-47cb-9931-31f838c5d2cb None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:05.404610 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796b3e89-b487-47cb-9931-31f838c5d2cb None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:05.404893 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796b3e89-b487-47cb-9931-31f838c5d2cb None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:05.405669 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796b3e89-b487-47cb-9931-31f838c5d2cb None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:55:05.405981 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796b3e89-b487-47cb-9931-31f838c5d2cb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:05.407453 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796b3e89-b487-47cb-9931-31f838c5d2cb None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:05.416634 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 242/480] 10.222.0.46 () {56 vars in 1111 bytes} [Thu May 26 14:55:05 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:55:06.838747 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f40e0d5-ee29-4aa1-b02d-3eace0c4e5fe None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:06.840313 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f40e0d5-ee29-4aa1-b02d-3eace0c4e5fe None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:06.840802 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f40e0d5-ee29-4aa1-b02d-3eace0c4e5fe None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:06.842676 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 239/481] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 14:55:06 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 14:55:06.848571 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acece15-bc22-4570-a490-151f46fe0534 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:06.849479 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acece15-bc22-4570-a490-151f46fe0534 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:06.849977 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acece15-bc22-4570-a490-151f46fe0534 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:06.868085 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-9acece15-bc22-4570-a490-151f46fe0534 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:55:06.869680 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-9acece15-bc22-4570-a490-151f46fe0534 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:55:06.969643 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 243/482] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 14:55:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 14:55:06.986919 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f40e0d5-ee29-4aa1-b02d-3eace0c4e5fe None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:07.021662 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:07.097361 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f40e0d5-ee29-4aa1-b02d-3eace0c4e5fe None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:07.145021 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d467bffc-84a3-4091-8b9a-5e5ad45bc7bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:07.146260 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d467bffc-84a3-4091-8b9a-5e5ad45bc7bf None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:07.146763 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d467bffc-84a3-4091-8b9a-5e5ad45bc7bf None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:07.147245 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d467bffc-84a3-4091-8b9a-5e5ad45bc7bf None admin] PATH_INFO: `/v3/roles/service` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:07.165011 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd764c8-4d6a-4da6-b2be-47bd9173f265 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:07.165640 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd764c8-4d6a-4da6-b2be-47bd9173f265 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:07.166122 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd764c8-4d6a-4da6-b2be-47bd9173f265 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:07.168020 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 244/483] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 14:55:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 14:55:07.169778 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d467bffc-84a3-4091-8b9a-5e5ad45bc7bf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 14:55:07.178212 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-d467bffc-84a3-4091-8b9a-5e5ad45bc7bf None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 26 14:55:07.178212 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:55:07.178212 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:55:07.178212 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:55:07.178212 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:55:07.178212 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:55:07.178212 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:55:07.178212 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:55:07.178212 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:55:07.178212 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:55:07.178212 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:55:07.178212 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:55:07.178212 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 14:55:07.178212 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 14:55:07.178212 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 14:55:07.178212 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 14:55:07.179133 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 14:55:07.179133 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 14:55:07.179133 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 14:55:07.179133 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 14:55:07.179133 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:55:07.179133 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:55:07.179133 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:55:07.179133 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:55:07.179133 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:55:07.179133 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:55:07.179133 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:55:07.179133 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 26 14:55:07.179133 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:55:07.179133 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 26 14:55:07.179133 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:55:07.179133 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:55:07.179929 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:55:07.179929 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 26 14:55:07.179929 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:55:07.179929 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 26 14:55:07.179929 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 14:55:07.179929 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 14:55:07.179929 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 14:55:07.179929 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 14:55:07.179929 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 14:55:07.179929 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 14:55:07.179929 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 26 14:55:07.179929 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 26 14:55:07.180536 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece32a36-f55c-452e-8efb-b4d37ab36339 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:07.180536 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece32a36-f55c-452e-8efb-b4d37ab36339 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:07.180536 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece32a36-f55c-452e-8efb-b4d37ab36339 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:07.187750 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 240/484] 10.222.0.46 () {56 vars in 1102 bytes} [Thu May 26 14:55:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:55:07.189190 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d467bffc-84a3-4091-8b9a-5e5ad45bc7bf None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:07.205087 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-ece32a36-f55c-452e-8efb-b4d37ab36339 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:55:07.206456 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-ece32a36-f55c-452e-8efb-b4d37ab36339 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:55:07.220519 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:07.309665 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:07.320476 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d467bffc-84a3-4091-8b9a-5e5ad45bc7bf None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:07.332051 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 245/485] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 14:55:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 14:55:07.358856 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4114e0f-20bf-4538-9bc1-6869c43a5149 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:07.359891 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4114e0f-20bf-4538-9bc1-6869c43a5149 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:07.360369 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4114e0f-20bf-4538-9bc1-6869c43a5149 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:07.360833 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4114e0f-20bf-4538-9bc1-6869c43a5149 None admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:07.361802 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4114e0f-20bf-4538-9bc1-6869c43a5149 None admin] RBAC: Adding query filter params (name=service) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:55:07.362302 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4114e0f-20bf-4538-9bc1-6869c43a5149 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:07.363881 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4114e0f-20bf-4538-9bc1-6869c43a5149 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:07.372899 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece32a36-f55c-452e-8efb-b4d37ab36339 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:07.374092 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 241/486] 10.222.0.46 () {56 vars in 1103 bytes} [Thu May 26 14:55:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 14:55:07.382063 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4114e0f-20bf-4538-9bc1-6869c43a5149 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:07.479061 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece32a36-f55c-452e-8efb-b4d37ab36339 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:07.520082 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2110125b-75d9-493c-ada0-5e560a02320c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:07.520877 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2110125b-75d9-493c-ada0-5e560a02320c None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:07.521162 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2110125b-75d9-493c-ada0-5e560a02320c None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:07.521439 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2110125b-75d9-493c-ada0-5e560a02320c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:07.540468 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2110125b-75d9-493c-ada0-5e560a02320c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 14:55:07.548895 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-2110125b-75d9-493c-ada0-5e560a02320c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 26 14:55:07.548895 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:55:07.548895 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:55:07.548895 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:55:07.548895 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:55:07.548895 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:55:07.548895 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:55:07.548895 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:55:07.548895 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:55:07.548895 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:55:07.548895 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:55:07.548895 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:55:07.548895 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 14:55:07.548895 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 14:55:07.548895 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 14:55:07.548895 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 14:55:07.549824 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 14:55:07.549824 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 14:55:07.549824 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 14:55:07.549824 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 14:55:07.549824 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:55:07.549824 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:55:07.549824 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:55:07.549824 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:55:07.549824 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:55:07.549824 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:55:07.549824 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:55:07.549824 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 26 14:55:07.549824 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:55:07.549824 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 26 14:55:07.549824 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:55:07.549824 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:55:07.550627 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:55:07.550627 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 26 14:55:07.550627 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:55:07.550627 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 26 14:55:07.550627 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 14:55:07.550627 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 14:55:07.550627 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 14:55:07.550627 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 14:55:07.550627 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 14:55:07.550627 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 14:55:07.550627 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 26 14:55:07.550627 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 26 14:55:07.556862 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 246/487] 10.222.0.46 () {56 vars in 1102 bytes} [Thu May 26 14:55:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 14:55:07.566230 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2110125b-75d9-493c-ada0-5e560a02320c None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:07.575487 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4114e0f-20bf-4538-9bc1-6869c43a5149 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:07.581839 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:07.626956 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e3ee3a-cd05-4467-9eef-2eb1e33faa3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:07.627989 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e3ee3a-cd05-4467-9eef-2eb1e33faa3f None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:07.628432 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e3ee3a-cd05-4467-9eef-2eb1e33faa3f None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:07.628857 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e3ee3a-cd05-4467-9eef-2eb1e33faa3f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:07.641073 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e3ee3a-cd05-4467-9eef-2eb1e33faa3f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:07.642892 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e3ee3a-cd05-4467-9eef-2eb1e33faa3f None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:07.662295 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-e0e3ee3a-cd05-4467-9eef-2eb1e33faa3f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 14:55:07.662295 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:55:07.662295 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:55:07.662295 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:55:07.662295 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:55:07.662295 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:55:07.662295 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:55:07.662295 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:55:07.662295 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:55:07.662295 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:55:07.662295 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:55:07.662295 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:55:07.662295 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 14:55:07.662295 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 14:55:07.662295 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 14:55:07.663217 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 14:55:07.663217 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:55:07.663217 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:55:07.663217 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:55:07.663217 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:55:07.663217 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:55:07.663217 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:55:07.663217 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:55:07.663217 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 26 14:55:07.663217 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:55:07.663217 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 26 14:55:07.663217 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:55:07.663217 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:55:07.663217 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:55:07.663217 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 26 14:55:07.663217 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:55:07.663217 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 26 14:55:07.664071 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 14:55:07.664071 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 14:55:07.664071 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 14:55:07.664071 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 26 14:55:07.670529 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 242/488] 10.222.0.46 () {56 vars in 1108 bytes} [Thu May 26 14:55:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:55:07.672073 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b16e12-7ddd-440b-bb87-6ab1a6372818 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:07.672565 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b16e12-7ddd-440b-bb87-6ab1a6372818 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:07.673029 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b16e12-7ddd-440b-bb87-6ab1a6372818 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:07.674982 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 243/489] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 14:55:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 14:55:07.676292 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b16e12-7ddd-440b-bb87-6ab1a6372818 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:07.694137 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:07.730249 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2110125b-75d9-493c-ada0-5e560a02320c None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:07.781585 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be2a2e9-311a-4555-9201-49996544e4b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:07.782366 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be2a2e9-311a-4555-9201-49996544e4b2 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:07.782654 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be2a2e9-311a-4555-9201-49996544e4b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:07.782931 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be2a2e9-311a-4555-9201-49996544e4b2 None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:07.783811 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be2a2e9-311a-4555-9201-49996544e4b2 None admin] RBAC: Adding query filter params (name=service) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:55:07.784092 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be2a2e9-311a-4555-9201-49996544e4b2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:07.785604 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be2a2e9-311a-4555-9201-49996544e4b2 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:07.799094 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 247/490] 10.222.0.46 () {56 vars in 1103 bytes} [Thu May 26 14:55:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:55:07.800588 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7aec4a-0972-4aa9-8228-78daf7559f44 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:07.800889 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7aec4a-0972-4aa9-8228-78daf7559f44 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:07.801177 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7aec4a-0972-4aa9-8228-78daf7559f44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:07.805821 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b16e12-7ddd-440b-bb87-6ab1a6372818 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:07.819363 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-9b7aec4a-0972-4aa9-8228-78daf7559f44 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:55:07.820557 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-9b7aec4a-0972-4aa9-8228-78daf7559f44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:55:07.844778 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2a50b7-f04a-4ee9-a302-a9ee39e2733c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:07.846211 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2a50b7-f04a-4ee9-a302-a9ee39e2733c None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:07.846708 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2a50b7-f04a-4ee9-a302-a9ee39e2733c None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:07.847322 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2a50b7-f04a-4ee9-a302-a9ee39e2733c None admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:07.848456 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2a50b7-f04a-4ee9-a302-a9ee39e2733c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:55:07.848983 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2a50b7-f04a-4ee9-a302-a9ee39e2733c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:07.850698 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2a50b7-f04a-4ee9-a302-a9ee39e2733c None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:07.868672 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 244/491] 10.222.0.46 () {56 vars in 1109 bytes} [Thu May 26 14:55:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:55:07.869962 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2a50b7-f04a-4ee9-a302-a9ee39e2733c None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:07.890923 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:07.897589 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 248/492] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 14:55:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 14:55:07.898643 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7aec4a-0972-4aa9-8228-78daf7559f44 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:07.951019 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:07.991087 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2a50b7-f04a-4ee9-a302-a9ee39e2733c None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:08.028705 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7aec4a-0972-4aa9-8228-78daf7559f44 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:08.031110 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76675050-ae8b-413b-b268-9d1f34a41b53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:08.032148 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76675050-ae8b-413b-b268-9d1f34a41b53 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:08.032660 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76675050-ae8b-413b-b268-9d1f34a41b53 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:08.033153 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76675050-ae8b-413b-b268-9d1f34a41b53 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:08.044519 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76675050-ae8b-413b-b268-9d1f34a41b53 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:08.046480 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76675050-ae8b-413b-b268-9d1f34a41b53 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:08.065892 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c982f20b-fc19-407d-a930-da6fcdc20e9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:08.066786 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c982f20b-fc19-407d-a930-da6fcdc20e9d None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:08.066978 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-76675050-ae8b-413b-b268-9d1f34a41b53 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 14:55:08.066978 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:55:08.066978 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:55:08.066978 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:55:08.066978 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:55:08.066978 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:55:08.066978 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:55:08.066978 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:55:08.066978 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:55:08.066978 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:55:08.066978 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:55:08.066978 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:55:08.066978 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 14:55:08.066978 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 14:55:08.066978 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 14:55:08.067849 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 14:55:08.067849 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:55:08.067849 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:55:08.067849 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:55:08.067849 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:55:08.067849 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:55:08.067849 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:55:08.067849 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:55:08.067849 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 26 14:55:08.067849 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:55:08.067849 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 26 14:55:08.067849 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:55:08.067849 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:55:08.067849 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:55:08.067849 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 26 14:55:08.067849 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:55:08.067849 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 26 14:55:08.068715 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 14:55:08.068715 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 14:55:08.068715 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 14:55:08.068715 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 26 14:55:08.068932 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c982f20b-fc19-407d-a930-da6fcdc20e9d None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:08.068932 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c982f20b-fc19-407d-a930-da6fcdc20e9d None admin] PATH_INFO: `/v3/users/glance` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:08.071369 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 245/493] 10.222.0.46 () {56 vars in 1108 bytes} [Thu May 26 14:55:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:55:08.072526 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76675050-ae8b-413b-b268-9d1f34a41b53 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:08.081304 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c982f20b-fc19-407d-a930-da6fcdc20e9d None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:08.082853 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c982f20b-fc19-407d-a930-da6fcdc20e9d None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:08.101098 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-c982f20b-fc19-407d-a930-da6fcdc20e9d None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 26 14:55:08.101098 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:55:08.101098 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:55:08.101098 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:55:08.101098 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:55:08.101098 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:55:08.101098 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:55:08.101098 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:55:08.101098 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:55:08.101098 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:55:08.101098 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:55:08.101098 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:55:08.101098 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 14:55:08.101098 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 14:55:08.101098 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 14:55:08.101098 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 14:55:08.102452 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:55:08.102452 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:55:08.102452 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 14:55:08.102452 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 14:55:08.102452 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 14:55:08.102452 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 14:55:08.102452 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:55:08.102452 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:55:08.102452 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:55:08.102452 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:55:08.102452 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:55:08.102452 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 26 14:55:08.102452 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:55:08.102452 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 26 14:55:08.102452 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:55:08.102452 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:55:08.102452 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:55:08.102452 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 26 14:55:08.105480 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:55:08.105480 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 26 14:55:08.105480 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 14:55:08.105480 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 14:55:08.105480 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 14:55:08.105480 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 14:55:08.105480 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 14:55:08.105480 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 14:55:08.105480 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 26 14:55:08.105480 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 26 14:55:08.108682 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 249/494] 10.222.0.46 () {56 vars in 1099 bytes} [Thu May 26 14:55:07 2022] GET /identity/v3/users/glance => generated 84 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:55:08.109548 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c982f20b-fc19-407d-a930-da6fcdc20e9d None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:08.170104 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:08.195934 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76675050-ae8b-413b-b268-9d1f34a41b53 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:08.213934 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:08.215158 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c982f20b-fc19-407d-a930-da6fcdc20e9d None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:08.232255 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d412ef75-a1ab-41e2-90a7-fb59dfe61cd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:08.233071 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d412ef75-a1ab-41e2-90a7-fb59dfe61cd9 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:08.233353 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d412ef75-a1ab-41e2-90a7-fb59dfe61cd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:08.233657 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d412ef75-a1ab-41e2-90a7-fb59dfe61cd9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:08.244508 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d412ef75-a1ab-41e2-90a7-fb59dfe61cd9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 14:55:08.252902 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-d412ef75-a1ab-41e2-90a7-fb59dfe61cd9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 26 14:55:08.252902 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:55:08.252902 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:55:08.252902 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:55:08.252902 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:55:08.252902 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:55:08.252902 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:55:08.252902 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:55:08.252902 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:55:08.252902 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:55:08.252902 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:55:08.252902 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:55:08.252902 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 14:55:08.252902 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 14:55:08.252902 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 14:55:08.252902 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 14:55:08.254010 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 14:55:08.254010 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 14:55:08.254010 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 14:55:08.254010 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 14:55:08.254010 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:55:08.254010 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:55:08.254010 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:55:08.254010 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:55:08.254010 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:55:08.254010 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:55:08.254010 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:55:08.254010 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 26 14:55:08.254010 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:55:08.254010 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 26 14:55:08.254010 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:55:08.254010 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:55:08.254820 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:55:08.254820 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 26 14:55:08.254820 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:55:08.254820 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 26 14:55:08.254820 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 14:55:08.254820 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 14:55:08.254820 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 14:55:08.254820 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 14:55:08.254820 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 14:55:08.254820 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 14:55:08.254820 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 26 14:55:08.254820 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 26 14:55:08.255681 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a070c418-273d-49a3-ae1f-b7e93c521133 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:08.255681 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a070c418-273d-49a3-ae1f-b7e93c521133 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:08.255681 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a070c418-273d-49a3-ae1f-b7e93c521133 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:08.255880 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 246/495] 10.222.0.46 () {56 vars in 1102 bytes} [Thu May 26 14:55:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:55:08.255880 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d412ef75-a1ab-41e2-90a7-fb59dfe61cd9 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:08.259071 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a070c418-273d-49a3-ae1f-b7e93c521133 None admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:08.260327 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a070c418-273d-49a3-ae1f-b7e93c521133 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:55:08.260327 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a070c418-273d-49a3-ae1f-b7e93c521133 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:08.262238 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a070c418-273d-49a3-ae1f-b7e93c521133 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:08.277649 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 250/496] 10.222.0.46 () {56 vars in 1109 bytes} [Thu May 26 14:55:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:55:08.277649 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62a37b2-ff13-4f9f-9a3a-a2782e24d839 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:08.277649 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62a37b2-ff13-4f9f-9a3a-a2782e24d839 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:08.277649 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62a37b2-ff13-4f9f-9a3a-a2782e24d839 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:08.278646 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 251/497] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 14:55:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 14:55:08.279655 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62a37b2-ff13-4f9f-9a3a-a2782e24d839 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:08.365204 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d412ef75-a1ab-41e2-90a7-fb59dfe61cd9 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:08.384386 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62a37b2-ff13-4f9f-9a3a-a2782e24d839 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:08.399480 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882441e0-1fa9-4c10-bfec-db958e1cc12f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:08.400145 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882441e0-1fa9-4c10-bfec-db958e1cc12f None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:08.400428 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882441e0-1fa9-4c10-bfec-db958e1cc12f None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:08.400708 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882441e0-1fa9-4c10-bfec-db958e1cc12f None admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:08.401573 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882441e0-1fa9-4c10-bfec-db958e1cc12f None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:55:08.401718 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:08.402540 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882441e0-1fa9-4c10-bfec-db958e1cc12f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:08.404224 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882441e0-1fa9-4c10-bfec-db958e1cc12f None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:08.420102 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7572c238-d13b-498b-9c9c-a635f78a3cca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:08.420792 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7572c238-d13b-498b-9c9c-a635f78a3cca None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:08.421073 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7572c238-d13b-498b-9c9c-a635f78a3cca None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:08.421346 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7572c238-d13b-498b-9c9c-a635f78a3cca None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:08.422199 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7572c238-d13b-498b-9c9c-a635f78a3cca None admin] RBAC: Adding query filter params (name=service) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:55:08.422416 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7572c238-d13b-498b-9c9c-a635f78a3cca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:08.423782 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7572c238-d13b-498b-9c9c-a635f78a3cca None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:08.432139 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 252/498] 10.222.0.46 () {56 vars in 1103 bytes} [Thu May 26 14:55:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:55:08.433560 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa16be59-1b9d-407c-8ae3-c291bb668e2f None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:08.433851 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa16be59-1b9d-407c-8ae3-c291bb668e2f None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:08.434125 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa16be59-1b9d-407c-8ae3-c291bb668e2f None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:08.435626 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 253/499] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 14:55:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 14:55:08.436201 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa16be59-1b9d-407c-8ae3-c291bb668e2f None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:08.447152 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 247/500] 10.222.0.46 () {56 vars in 1137 bytes} [Thu May 26 14:55:08 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:55:08.447152 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754754d6-f921-45f7-892e-12778ff8f432 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:08.447152 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754754d6-f921-45f7-892e-12778ff8f432 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:08.447152 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754754d6-f921-45f7-892e-12778ff8f432 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:08.462963 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-754754d6-f921-45f7-892e-12778ff8f432 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:55:08.464225 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-754754d6-f921-45f7-892e-12778ff8f432 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:55:08.492886 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:08.532806 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 248/501] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 14:55:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 14:55:08.534027 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754754d6-f921-45f7-892e-12778ff8f432 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:08.546549 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa16be59-1b9d-407c-8ae3-c291bb668e2f None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:08.581943 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93f9064-be09-45fb-86c2-b2d9920ccea0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:08.582771 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93f9064-be09-45fb-86c2-b2d9920ccea0 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:08.583056 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93f9064-be09-45fb-86c2-b2d9920ccea0 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:08.583337 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93f9064-be09-45fb-86c2-b2d9920ccea0 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:08.591019 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93f9064-be09-45fb-86c2-b2d9920ccea0 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:08.592450 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a93f9064-be09-45fb-86c2-b2d9920ccea0 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:08.606523 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-a93f9064-be09-45fb-86c2-b2d9920ccea0 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 26 14:55:08.606523 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:55:08.606523 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:55:08.606523 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:55:08.606523 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:55:08.606523 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:55:08.606523 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:55:08.606523 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:55:08.606523 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:55:08.606523 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:55:08.606523 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:55:08.606523 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:55:08.606523 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 14:55:08.606523 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 14:55:08.606523 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 14:55:08.606523 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 14:55:08.607483 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:55:08.607483 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:55:08.607483 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 14:55:08.607483 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 14:55:08.607483 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 14:55:08.607483 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 14:55:08.607483 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:55:08.607483 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:55:08.607483 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:55:08.607483 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:55:08.607483 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:55:08.607483 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 26 14:55:08.607483 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:55:08.607483 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 26 14:55:08.607483 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:55:08.607483 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:55:08.607483 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:55:08.607483 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 26 14:55:08.608372 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:55:08.608372 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 26 14:55:08.608372 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 14:55:08.608372 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 14:55:08.608372 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 14:55:08.608372 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 14:55:08.608372 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 14:55:08.608372 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 14:55:08.608372 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 26 14:55:08.608372 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 26 14:55:08.608372 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 254/502] 10.222.0.46 () {56 vars in 1093 bytes} [Thu May 26 14:55:08 2022] GET /identity/v3/users/nova => generated 82 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:55:08.610741 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4dea6c-ff77-45b6-b0ad-390f757e8843 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:08.611088 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4dea6c-ff77-45b6-b0ad-390f757e8843 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:08.611381 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4dea6c-ff77-45b6-b0ad-390f757e8843 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:08.629147 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-8c4dea6c-ff77-45b6-b0ad-390f757e8843 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:55:08.630722 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-8c4dea6c-ff77-45b6-b0ad-390f757e8843 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:55:08.643285 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754754d6-f921-45f7-892e-12778ff8f432 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:08.665542 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:08.680030 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1342efd3-d813-495a-a3e1-7ddb176b504c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:08.680822 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1342efd3-d813-495a-a3e1-7ddb176b504c None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:08.681096 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1342efd3-d813-495a-a3e1-7ddb176b504c None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:08.681364 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1342efd3-d813-495a-a3e1-7ddb176b504c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:08.691727 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1342efd3-d813-495a-a3e1-7ddb176b504c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:08.693187 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1342efd3-d813-495a-a3e1-7ddb176b504c None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:08.702060 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 255/503] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 14:55:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 14:55:08.702781 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4dea6c-ff77-45b6-b0ad-390f757e8843 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:08.718770 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-1342efd3-d813-495a-a3e1-7ddb176b504c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 14:55:08.718770 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:55:08.718770 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:55:08.718770 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:55:08.718770 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:55:08.718770 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:55:08.718770 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:55:08.718770 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:55:08.718770 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:55:08.718770 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:55:08.718770 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:55:08.718770 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:55:08.718770 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 14:55:08.718770 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 14:55:08.718770 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 14:55:08.719624 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 14:55:08.719624 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:55:08.719624 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:55:08.719624 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:55:08.719624 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:55:08.719624 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:55:08.719624 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:55:08.719624 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:55:08.719624 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 26 14:55:08.719624 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:55:08.719624 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 26 14:55:08.719624 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:55:08.719624 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:55:08.719624 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:55:08.719624 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 26 14:55:08.719624 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:55:08.719624 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 26 14:55:08.720466 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 14:55:08.720466 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 14:55:08.720466 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 14:55:08.720466 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 26 14:55:08.720466 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 249/504] 10.222.0.46 () {56 vars in 1108 bytes} [Thu May 26 14:55:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:55:08.720745 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1342efd3-d813-495a-a3e1-7ddb176b504c None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:08.740062 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:08.806815 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4dea6c-ff77-45b6-b0ad-390f757e8843 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:08.821955 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1342efd3-d813-495a-a3e1-7ddb176b504c None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:08.841182 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4986d08-47c5-4dab-9775-035ed2aa062f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:08.841867 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4986d08-47c5-4dab-9775-035ed2aa062f None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:08.842179 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4986d08-47c5-4dab-9775-035ed2aa062f None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:08.842462 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4986d08-47c5-4dab-9775-035ed2aa062f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:08.855917 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03727ea-1ff1-423d-8ff8-801be6104825 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:08.856704 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03727ea-1ff1-423d-8ff8-801be6104825 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:08.856986 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03727ea-1ff1-423d-8ff8-801be6104825 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:08.857259 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03727ea-1ff1-423d-8ff8-801be6104825 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:08.867499 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03727ea-1ff1-423d-8ff8-801be6104825 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 14:55:08.875812 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-e03727ea-1ff1-423d-8ff8-801be6104825 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 26 14:55:08.875812 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:55:08.875812 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:55:08.875812 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:55:08.875812 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:55:08.875812 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:55:08.875812 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:55:08.875812 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:55:08.875812 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:55:08.875812 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:55:08.875812 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:55:08.875812 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:55:08.875812 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 14:55:08.875812 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 14:55:08.875812 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 14:55:08.875812 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 14:55:08.876695 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 14:55:08.876695 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 14:55:08.876695 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 14:55:08.876695 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 14:55:08.876695 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:55:08.876695 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:55:08.876695 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:55:08.876695 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:55:08.876695 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:55:08.876695 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:55:08.876695 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:55:08.876695 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 26 14:55:08.876695 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:55:08.876695 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 26 14:55:08.876695 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:55:08.876695 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:55:08.877509 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:55:08.877509 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 26 14:55:08.877509 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:55:08.877509 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 26 14:55:08.877509 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 14:55:08.877509 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 14:55:08.877509 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 14:55:08.877509 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 14:55:08.877509 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 14:55:08.877509 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 14:55:08.877509 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 26 14:55:08.877509 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 26 14:55:08.877509 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 250/505] 10.222.0.46 () {56 vars in 1102 bytes} [Thu May 26 14:55:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:55:08.878318 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03727ea-1ff1-423d-8ff8-801be6104825 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:08.886921 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:08.915542 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4986d08-47c5-4dab-9775-035ed2aa062f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:08.916994 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4986d08-47c5-4dab-9775-035ed2aa062f None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:08.957316 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:08.979177 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03727ea-1ff1-423d-8ff8-801be6104825 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:09.013758 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58278c1e-1cfb-4f0a-af73-10ff894d1175 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:09.014425 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58278c1e-1cfb-4f0a-af73-10ff894d1175 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:09.014707 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58278c1e-1cfb-4f0a-af73-10ff894d1175 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:09.014984 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58278c1e-1cfb-4f0a-af73-10ff894d1175 None admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:09.015816 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58278c1e-1cfb-4f0a-af73-10ff894d1175 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:55:09.016126 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58278c1e-1cfb-4f0a-af73-10ff894d1175 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:09.017594 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58278c1e-1cfb-4f0a-af73-10ff894d1175 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:09.018231 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 256/506] 10.222.0.46 () {58 vars in 1282 bytes} [Thu May 26 14:55:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 14:55:09.019143 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4986d08-47c5-4dab-9775-035ed2aa062f None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:09.040130 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 251/507] 10.222.0.46 () {56 vars in 1133 bytes} [Thu May 26 14:55:08 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:55:09.040130 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58278c1e-1cfb-4f0a-af73-10ff894d1175 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:09.091142 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:09.122642 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4986d08-47c5-4dab-9775-035ed2aa062f None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:09.139578 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58278c1e-1cfb-4f0a-af73-10ff894d1175 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:09.156017 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02174def-a707-4342-a45d-709acdf10bf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:09.156670 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02174def-a707-4342-a45d-709acdf10bf3 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:09.156948 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02174def-a707-4342-a45d-709acdf10bf3 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:09.157217 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02174def-a707-4342-a45d-709acdf10bf3 None admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:09.157967 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02174def-a707-4342-a45d-709acdf10bf3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:55:09.158269 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02174def-a707-4342-a45d-709acdf10bf3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:09.159817 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02174def-a707-4342-a45d-709acdf10bf3 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:09.171039 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 257/508] 10.222.0.46 () {56 vars in 1109 bytes} [Thu May 26 14:55:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:55:09.171813 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02174def-a707-4342-a45d-709acdf10bf3 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:09.173284 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cf52eb-6e1f-4747-a545-151c7670debc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:09.174152 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cf52eb-6e1f-4747-a545-151c7670debc None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:09.174446 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cf52eb-6e1f-4747-a545-151c7670debc None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:09.174723 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cf52eb-6e1f-4747-a545-151c7670debc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:09.177593 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:09.186663 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8cf52eb-6e1f-4747-a545-151c7670debc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:09.188129 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8cf52eb-6e1f-4747-a545-151c7670debc None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:09.206101 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-e8cf52eb-6e1f-4747-a545-151c7670debc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 14:55:09.206101 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:55:09.206101 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:55:09.206101 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:55:09.206101 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:55:09.206101 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:55:09.206101 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:55:09.206101 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:55:09.206101 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:55:09.206101 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:55:09.206101 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:55:09.206101 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:55:09.206101 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 14:55:09.206101 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 14:55:09.206101 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 14:55:09.206964 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 14:55:09.206964 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:55:09.206964 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:55:09.206964 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:55:09.206964 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:55:09.206964 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:55:09.206964 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:55:09.206964 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:55:09.206964 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 26 14:55:09.206964 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:55:09.206964 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 26 14:55:09.206964 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:55:09.206964 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:55:09.206964 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:55:09.206964 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 26 14:55:09.206964 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:55:09.206964 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 26 14:55:09.208011 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 14:55:09.208011 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 14:55:09.208011 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 14:55:09.208011 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 26 14:55:09.208011 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 252/509] 10.222.0.46 () {56 vars in 1108 bytes} [Thu May 26 14:55:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:55:09.209270 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cf52eb-6e1f-4747-a545-151c7670debc None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:09.274767 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02174def-a707-4342-a45d-709acdf10bf3 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:09.303563 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:09.306921 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cf52eb-6e1f-4747-a545-151c7670debc None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:09.310424 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112e94cc-4ea4-435d-8fd8-d9b54ec3c3d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:09.311104 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112e94cc-4ea4-435d-8fd8-d9b54ec3c3d5 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:09.311392 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112e94cc-4ea4-435d-8fd8-d9b54ec3c3d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:09.311688 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112e94cc-4ea4-435d-8fd8-d9b54ec3c3d5 None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:09.312436 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112e94cc-4ea4-435d-8fd8-d9b54ec3c3d5 None admin] RBAC: Adding query filter params (name=service) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:55:09.312747 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112e94cc-4ea4-435d-8fd8-d9b54ec3c3d5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:09.314151 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112e94cc-4ea4-435d-8fd8-d9b54ec3c3d5 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:09.322868 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 258/510] 10.222.0.46 () {56 vars in 1103 bytes} [Thu May 26 14:55:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:55:09.323715 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112e94cc-4ea4-435d-8fd8-d9b54ec3c3d5 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:09.348563 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b92b86-597f-4240-b443-2566d42d0397 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:09.349287 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b92b86-597f-4240-b443-2566d42d0397 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:09.349603 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b92b86-597f-4240-b443-2566d42d0397 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:09.349888 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b92b86-597f-4240-b443-2566d42d0397 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:09.360341 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b92b86-597f-4240-b443-2566d42d0397 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:09.361831 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b92b86-597f-4240-b443-2566d42d0397 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:09.381081 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-d0b92b86-597f-4240-b443-2566d42d0397 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 14:55:09.381081 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:55:09.381081 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:55:09.381081 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:55:09.381081 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:55:09.381081 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:55:09.381081 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:55:09.381081 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:55:09.381081 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:55:09.381081 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:55:09.381081 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:55:09.381081 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:55:09.381081 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 14:55:09.381081 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 14:55:09.381081 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 14:55:09.382018 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 14:55:09.382018 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:55:09.382018 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:55:09.382018 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:55:09.382018 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:55:09.382018 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:55:09.382018 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:55:09.382018 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:55:09.382018 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 26 14:55:09.382018 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:55:09.382018 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 26 14:55:09.382018 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:55:09.382018 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:55:09.382018 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:55:09.382018 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 26 14:55:09.382018 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:55:09.382018 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 26 14:55:09.382872 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 14:55:09.382872 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 14:55:09.382872 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 14:55:09.382872 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 26 14:55:09.382872 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 253/511] 10.222.0.46 () {56 vars in 1108 bytes} [Thu May 26 14:55:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:55:09.383139 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b92b86-597f-4240-b443-2566d42d0397 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:09.424609 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:09.431989 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112e94cc-4ea4-435d-8fd8-d9b54ec3c3d5 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:09.466969 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccc9ea7-1224-4be6-a44f-66727d7a831b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:09.467646 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccc9ea7-1224-4be6-a44f-66727d7a831b None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:09.467930 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccc9ea7-1224-4be6-a44f-66727d7a831b None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:09.468206 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccc9ea7-1224-4be6-a44f-66727d7a831b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:09.488964 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b92b86-597f-4240-b443-2566d42d0397 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:09.522264 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b82c92-dc98-4d7c-8341-18c1d3c8b1f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:09.523092 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b82c92-dc98-4d7c-8341-18c1d3c8b1f2 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:09.523378 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b82c92-dc98-4d7c-8341-18c1d3c8b1f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:09.523655 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b82c92-dc98-4d7c-8341-18c1d3c8b1f2 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:09.528520 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:09.532779 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b82c92-dc98-4d7c-8341-18c1d3c8b1f2 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:09.534282 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b82c92-dc98-4d7c-8341-18c1d3c8b1f2 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:09.539700 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ccc9ea7-1224-4be6-a44f-66727d7a831b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:09.541273 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ccc9ea7-1224-4be6-a44f-66727d7a831b None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:09.550888 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-56b82c92-dc98-4d7c-8341-18c1d3c8b1f2 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 26 14:55:09.550888 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:55:09.550888 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:55:09.550888 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:55:09.550888 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:55:09.550888 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:55:09.550888 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:55:09.550888 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:55:09.550888 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:55:09.550888 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:55:09.550888 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:55:09.550888 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:55:09.550888 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 14:55:09.550888 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 14:55:09.550888 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 14:55:09.550888 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 14:55:09.551820 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:55:09.551820 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:55:09.551820 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 14:55:09.551820 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 14:55:09.551820 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 14:55:09.551820 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 14:55:09.551820 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:55:09.551820 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:55:09.551820 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:55:09.551820 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:55:09.551820 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:55:09.551820 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 26 14:55:09.551820 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:55:09.551820 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 26 14:55:09.551820 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:55:09.551820 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:55:09.551820 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:55:09.551820 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 26 14:55:09.552698 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:55:09.552698 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 26 14:55:09.552698 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 14:55:09.552698 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 14:55:09.552698 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 14:55:09.552698 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 14:55:09.552698 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 14:55:09.552698 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 14:55:09.552698 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 26 14:55:09.552698 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 26 14:55:09.552698 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 254/512] 10.222.0.46 () {56 vars in 1099 bytes} [Thu May 26 14:55:09 2022] GET /identity/v3/users/cinder => generated 84 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:55:09.553274 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b82c92-dc98-4d7c-8341-18c1d3c8b1f2 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:09.649346 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 259/513] 10.222.0.46 () {58 vars in 1282 bytes} [Thu May 26 14:55:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 14:55:09.650228 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccc9ea7-1224-4be6-a44f-66727d7a831b None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:09.666034 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:09.669778 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b82c92-dc98-4d7c-8341-18c1d3c8b1f2 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:09.706387 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fa1265-eb92-4849-80bd-6ea40513d48f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:09.707045 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fa1265-eb92-4849-80bd-6ea40513d48f None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:09.707330 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fa1265-eb92-4849-80bd-6ea40513d48f None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:09.707607 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fa1265-eb92-4849-80bd-6ea40513d48f None admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:09.708325 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88fa1265-eb92-4849-80bd-6ea40513d48f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:55:09.708629 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88fa1265-eb92-4849-80bd-6ea40513d48f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:09.710063 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88fa1265-eb92-4849-80bd-6ea40513d48f None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:09.722090 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 255/514] 10.222.0.46 () {56 vars in 1109 bytes} [Thu May 26 14:55:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:55:09.722969 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fa1265-eb92-4849-80bd-6ea40513d48f None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:09.745294 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:09.755603 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccc9ea7-1224-4be6-a44f-66727d7a831b None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:09.791484 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408ae864-0936-4f08-8e56-ecac30c16dfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:09.792280 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408ae864-0936-4f08-8e56-ecac30c16dfc None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:09.793087 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408ae864-0936-4f08-8e56-ecac30c16dfc None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:09.793369 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408ae864-0936-4f08-8e56-ecac30c16dfc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:09.803601 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408ae864-0936-4f08-8e56-ecac30c16dfc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:09.805094 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408ae864-0936-4f08-8e56-ecac30c16dfc None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:09.823777 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-408ae864-0936-4f08-8e56-ecac30c16dfc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 14:55:09.823777 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:55:09.823777 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:55:09.823777 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:55:09.823777 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:55:09.823777 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:55:09.823777 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:55:09.823777 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:55:09.823777 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:55:09.823777 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:55:09.823777 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:55:09.823777 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:55:09.823777 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 14:55:09.823777 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 14:55:09.823777 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 14:55:09.824677 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 14:55:09.824677 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:55:09.824677 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:55:09.824677 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:55:09.824677 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:55:09.824677 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:55:09.824677 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:55:09.824677 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:55:09.824677 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 26 14:55:09.824677 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:55:09.824677 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 26 14:55:09.824677 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:55:09.824677 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:55:09.824677 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:55:09.824677 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 26 14:55:09.824677 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:55:09.824677 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 26 14:55:09.826003 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 14:55:09.826003 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 14:55:09.826003 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 14:55:09.826003 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 26 14:55:09.826003 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 260/515] 10.222.0.46 () {56 vars in 1108 bytes} [Thu May 26 14:55:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:55:09.826426 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408ae864-0936-4f08-8e56-ecac30c16dfc None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:09.840781 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fa1265-eb92-4849-80bd-6ea40513d48f None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:09.876475 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb6020d-084e-4de8-a765-7540bd47ea54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:09.877233 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb6020d-084e-4de8-a765-7540bd47ea54 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:09.877475 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb6020d-084e-4de8-a765-7540bd47ea54 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:09.877766 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb6020d-084e-4de8-a765-7540bd47ea54 None admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:09.878588 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb6020d-084e-4de8-a765-7540bd47ea54 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:55:09.878964 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb6020d-084e-4de8-a765-7540bd47ea54 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:09.880630 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb6020d-084e-4de8-a765-7540bd47ea54 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:09.897708 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 256/516] 10.222.0.46 () {56 vars in 1109 bytes} [Thu May 26 14:55:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:55:09.897708 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb6020d-084e-4de8-a765-7540bd47ea54 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:09.929615 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:09.948457 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408ae864-0936-4f08-8e56-ecac30c16dfc None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:09.991229 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f33697d-f6dc-4860-b154-0c6ea3f9e435 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:09.991906 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f33697d-f6dc-4860-b154-0c6ea3f9e435 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:09.992187 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f33697d-f6dc-4860-b154-0c6ea3f9e435 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:09.992461 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f33697d-f6dc-4860-b154-0c6ea3f9e435 None admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:09.993556 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f33697d-f6dc-4860-b154-0c6ea3f9e435 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:55:09.993875 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f33697d-f6dc-4860-b154-0c6ea3f9e435 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:09.995418 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f33697d-f6dc-4860-b154-0c6ea3f9e435 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:10.011730 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb6020d-084e-4de8-a765-7540bd47ea54 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:10.034096 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 261/517] 10.222.0.46 () {56 vars in 1137 bytes} [Thu May 26 14:55:09 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:55:10.034922 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f33697d-f6dc-4860-b154-0c6ea3f9e435 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:10.037619 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:10.049218 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516fec1f-5b12-42c8-a4b0-9f27c3353175 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:10.050035 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516fec1f-5b12-42c8-a4b0-9f27c3353175 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:10.050318 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516fec1f-5b12-42c8-a4b0-9f27c3353175 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:10.050591 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516fec1f-5b12-42c8-a4b0-9f27c3353175 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:10.060986 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516fec1f-5b12-42c8-a4b0-9f27c3353175 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:10.062492 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-516fec1f-5b12-42c8-a4b0-9f27c3353175 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:10.080708 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-516fec1f-5b12-42c8-a4b0-9f27c3353175 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 14:55:10.080708 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:55:10.080708 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:55:10.080708 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:55:10.080708 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:55:10.080708 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:55:10.080708 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:55:10.080708 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:55:10.080708 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:55:10.080708 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:55:10.080708 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:55:10.080708 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:55:10.080708 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 14:55:10.080708 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 14:55:10.080708 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 14:55:10.081779 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 14:55:10.081779 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:55:10.081779 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:55:10.081779 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:55:10.081779 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:55:10.081779 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:55:10.081779 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:55:10.081779 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:55:10.081779 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 26 14:55:10.081779 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:55:10.081779 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 26 14:55:10.081779 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:55:10.081779 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:55:10.081779 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:55:10.081779 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 26 14:55:10.081779 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:55:10.081779 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 26 14:55:10.087355 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 14:55:10.087355 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 14:55:10.087355 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 14:55:10.087355 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 26 14:55:10.087355 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 257/518] 10.222.0.46 () {56 vars in 1108 bytes} [Thu May 26 14:55:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:55:10.087355 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516fec1f-5b12-42c8-a4b0-9f27c3353175 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:10.146104 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f33697d-f6dc-4860-b154-0c6ea3f9e435 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:10.169551 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:10.184605 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689decd7-9c87-492e-bd7a-6d8eabe607a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:10.185556 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689decd7-9c87-492e-bd7a-6d8eabe607a4 None admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:10.185840 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689decd7-9c87-492e-bd7a-6d8eabe607a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:10.186117 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689decd7-9c87-492e-bd7a-6d8eabe607a4 None admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:10.186848 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689decd7-9c87-492e-bd7a-6d8eabe607a4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:10.188284 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689decd7-9c87-492e-bd7a-6d8eabe607a4 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:10.200443 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516fec1f-5b12-42c8-a4b0-9f27c3353175 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:10.208480 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 262/519] 10.222.0.46 () {60 vars in 1132 bytes} [Thu May 26 14:55:10 2022] POST /identity/v3/users => generated 247 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 14:55:10.209553 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689decd7-9c87-492e-bd7a-6d8eabe607a4 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:10.240270 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbe953c-5413-4228-8bf4-664b67b17e8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:10.241099 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbe953c-5413-4228-8bf4-664b67b17e8f None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:10.241394 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbe953c-5413-4228-8bf4-664b67b17e8f None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:10.241701 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbe953c-5413-4228-8bf4-664b67b17e8f None admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:10.242440 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbe953c-5413-4228-8bf4-664b67b17e8f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:55:10.242746 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbe953c-5413-4228-8bf4-664b67b17e8f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:10.244179 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbe953c-5413-4228-8bf4-664b67b17e8f None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:10.262894 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 258/520] 10.222.0.46 () {56 vars in 1109 bytes} [Thu May 26 14:55:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:55:10.262894 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbe953c-5413-4228-8bf4-664b67b17e8f None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:10.282902 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:10.336782 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689decd7-9c87-492e-bd7a-6d8eabe607a4 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:10.376671 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7b0bf2-8172-4236-90fa-db69a86fb692 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:10.377510 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7b0bf2-8172-4236-90fa-db69a86fb692 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:10.377797 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7b0bf2-8172-4236-90fa-db69a86fb692 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:10.378073 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7b0bf2-8172-4236-90fa-db69a86fb692 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:10.383006 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbe953c-5413-4228-8bf4-664b67b17e8f None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:10.386964 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7b0bf2-8172-4236-90fa-db69a86fb692 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:10.388442 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7b0bf2-8172-4236-90fa-db69a86fb692 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:10.406320 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-fb7b0bf2-8172-4236-90fa-db69a86fb692 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 14:55:10.406320 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:55:10.406320 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:55:10.406320 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:55:10.406320 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:55:10.406320 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:55:10.406320 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:55:10.406320 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:55:10.406320 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:55:10.406320 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:55:10.406320 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:55:10.406320 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:55:10.406320 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 14:55:10.406320 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 14:55:10.406320 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 14:55:10.409070 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 14:55:10.409070 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:55:10.409070 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:55:10.409070 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:55:10.409070 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:55:10.409070 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:55:10.409070 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:55:10.409070 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:55:10.409070 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 26 14:55:10.409070 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:55:10.409070 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 26 14:55:10.409070 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:55:10.409070 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:55:10.409070 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:55:10.409070 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 26 14:55:10.409070 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:55:10.409070 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 26 14:55:10.411273 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 14:55:10.411273 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 14:55:10.411273 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 14:55:10.411273 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 14:55:10.411273 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 14:55:10.411273 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 14:55:10.411273 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 14:55:10.411273 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 26 14:55:10.412227 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 263/521] 10.222.0.46 () {56 vars in 1111 bytes} [Thu May 26 14:55:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:55:10.413149 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7b0bf2-8172-4236-90fa-db69a86fb692 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:10.423866 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:10.434925 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d092d49a-2dc3-43c4-804c-618ca7f74eee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:10.435663 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d092d49a-2dc3-43c4-804c-618ca7f74eee None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:10.435949 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d092d49a-2dc3-43c4-804c-618ca7f74eee None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:10.436222 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d092d49a-2dc3-43c4-804c-618ca7f74eee None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:10.519115 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d092d49a-2dc3-43c4-804c-618ca7f74eee None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:10.521087 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d092d49a-2dc3-43c4-804c-618ca7f74eee None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:10.529718 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7b0bf2-8172-4236-90fa-db69a86fb692 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:10.550735 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:10.570412 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d18a606-88ff-4a1d-9ab8-71975808dcb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:10.571163 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d18a606-88ff-4a1d-9ab8-71975808dcb7 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:10.571426 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d18a606-88ff-4a1d-9ab8-71975808dcb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:10.571630 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d18a606-88ff-4a1d-9ab8-71975808dcb7 None admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:10.572363 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d18a606-88ff-4a1d-9ab8-71975808dcb7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:55:10.572684 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d18a606-88ff-4a1d-9ab8-71975808dcb7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:10.574567 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d18a606-88ff-4a1d-9ab8-71975808dcb7 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:10.587162 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 264/522] 10.222.0.46 () {56 vars in 1109 bytes} [Thu May 26 14:55:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:55:10.592852 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d18a606-88ff-4a1d-9ab8-71975808dcb7 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:10.634463 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 259/523] 10.222.0.46 () {58 vars in 1282 bytes} [Thu May 26 14:55:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 14:55:10.638450 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d092d49a-2dc3-43c4-804c-618ca7f74eee None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:10.673501 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:10.714034 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d18a606-88ff-4a1d-9ab8-71975808dcb7 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:10.750112 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d092d49a-2dc3-43c4-804c-618ca7f74eee None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:10.752135 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651c3a3c-567a-4536-a571-3a84b32ada62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:10.752943 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651c3a3c-567a-4536-a571-3a84b32ada62 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:10.753295 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651c3a3c-567a-4536-a571-3a84b32ada62 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:10.753652 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651c3a3c-567a-4536-a571-3a84b32ada62 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:10.762824 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651c3a3c-567a-4536-a571-3a84b32ada62 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:10.764264 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651c3a3c-567a-4536-a571-3a84b32ada62 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:10.780492 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-651c3a3c-567a-4536-a571-3a84b32ada62 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 26 14:55:10.780492 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:55:10.780492 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:55:10.780492 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:55:10.780492 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:55:10.780492 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:55:10.780492 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:55:10.780492 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:55:10.780492 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:55:10.780492 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:55:10.780492 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:55:10.780492 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:55:10.780492 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 14:55:10.780492 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 14:55:10.780492 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 14:55:10.780492 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 14:55:10.781543 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:55:10.781543 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:55:10.781543 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 14:55:10.781543 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 14:55:10.781543 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 14:55:10.781543 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 14:55:10.781543 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:55:10.781543 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:55:10.781543 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:55:10.781543 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:55:10.781543 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:55:10.781543 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 26 14:55:10.781543 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:55:10.781543 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 26 14:55:10.781543 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:55:10.781543 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:55:10.781543 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:55:10.781543 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 26 14:55:10.782590 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:55:10.782590 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 26 14:55:10.782590 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 14:55:10.782590 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 14:55:10.782590 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 14:55:10.782590 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 14:55:10.782590 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 14:55:10.782590 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 14:55:10.782590 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 26 14:55:10.782590 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 26 14:55:10.782590 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 265/524] 10.222.0.46 () {56 vars in 1102 bytes} [Thu May 26 14:55:10 2022] GET /identity/v3/users/neutron => generated 85 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:55:10.782590 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651c3a3c-567a-4536-a571-3a84b32ada62 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:10.795392 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e44198-60bf-40ca-bb07-b678dc8bf249 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:10.796295 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e44198-60bf-40ca-bb07-b678dc8bf249 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:10.796413 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e44198-60bf-40ca-bb07-b678dc8bf249 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:10.796627 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e44198-60bf-40ca-bb07-b678dc8bf249 None admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:10.797427 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e44198-60bf-40ca-bb07-b678dc8bf249 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:55:10.797790 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e44198-60bf-40ca-bb07-b678dc8bf249 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:10.799253 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e44198-60bf-40ca-bb07-b678dc8bf249 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:10.825083 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 260/525] 10.222.0.46 () {56 vars in 1148 bytes} [Thu May 26 14:55:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:55:10.825083 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e44198-60bf-40ca-bb07-b678dc8bf249 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:10.834136 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:10.897142 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651c3a3c-567a-4536-a571-3a84b32ada62 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:10.916906 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:10.927523 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e44198-60bf-40ca-bb07-b678dc8bf249 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:10.934846 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e1b5da-990d-4177-890b-5fb7c46f94a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:10.936108 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e1b5da-990d-4177-890b-5fb7c46f94a8 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:10.936394 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e1b5da-990d-4177-890b-5fb7c46f94a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:10.936674 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e1b5da-990d-4177-890b-5fb7c46f94a8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:10.944813 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e1b5da-990d-4177-890b-5fb7c46f94a8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:10.946292 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e1b5da-990d-4177-890b-5fb7c46f94a8 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:10.961126 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-95e1b5da-990d-4177-890b-5fb7c46f94a8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 14:55:10.961126 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:55:10.961126 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:55:10.961126 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:55:10.961126 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:55:10.961126 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:55:10.961126 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:55:10.961126 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:55:10.961126 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:55:10.961126 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:55:10.961126 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:55:10.961126 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:55:10.961126 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 14:55:10.961126 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 14:55:10.961126 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 14:55:10.963099 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 14:55:10.963099 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:55:10.963099 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:55:10.963099 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:55:10.963099 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:55:10.963099 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:55:10.963099 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:55:10.963099 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:55:10.963099 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 26 14:55:10.963099 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:55:10.963099 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 26 14:55:10.963099 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:55:10.963099 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:55:10.963099 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:55:10.963099 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 26 14:55:10.963099 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:55:10.963099 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 26 14:55:10.964078 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 14:55:10.964078 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 14:55:10.964078 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 14:55:10.964078 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 14:55:10.964078 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 14:55:10.964078 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 14:55:10.964078 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 14:55:10.964078 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 26 14:55:10.964078 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 266/526] 10.222.0.46 () {56 vars in 1111 bytes} [Thu May 26 14:55:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:55:10.964078 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e1b5da-990d-4177-890b-5fb7c46f94a8 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:10.969506 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4141ee0d-9224-4703-bc16-537ff772802b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:10.970349 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4141ee0d-9224-4703-bc16-537ff772802b None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:10.970635 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4141ee0d-9224-4703-bc16-537ff772802b None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:10.970910 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4141ee0d-9224-4703-bc16-537ff772802b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:10.982524 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4141ee0d-9224-4703-bc16-537ff772802b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:10.984007 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4141ee0d-9224-4703-bc16-537ff772802b None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:11.004869 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-4141ee0d-9224-4703-bc16-537ff772802b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 14:55:11.004869 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:55:11.004869 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:55:11.004869 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:55:11.004869 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:55:11.004869 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:55:11.004869 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:55:11.004869 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:55:11.004869 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:55:11.004869 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:55:11.004869 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:55:11.004869 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:55:11.004869 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 14:55:11.004869 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 14:55:11.004869 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 14:55:11.006097 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 14:55:11.006097 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:55:11.006097 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:55:11.006097 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:55:11.006097 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:55:11.006097 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:55:11.006097 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:55:11.006097 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:55:11.006097 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 26 14:55:11.006097 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:55:11.006097 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 26 14:55:11.006097 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:55:11.006097 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:55:11.006097 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:55:11.006097 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 26 14:55:11.006097 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:55:11.006097 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 26 14:55:11.006942 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 14:55:11.006942 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 14:55:11.006942 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 14:55:11.006942 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 26 14:55:11.009990 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 261/527] 10.222.0.46 () {56 vars in 1108 bytes} [Thu May 26 14:55:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:55:11.011068 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4141ee0d-9224-4703-bc16-537ff772802b None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:11.074764 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:11.079010 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e1b5da-990d-4177-890b-5fb7c46f94a8 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:11.115722 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3641bb1-a457-4583-b5e1-87e2968122fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:11.116513 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3641bb1-a457-4583-b5e1-87e2968122fa None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:11.116838 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3641bb1-a457-4583-b5e1-87e2968122fa None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:11.117116 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3641bb1-a457-4583-b5e1-87e2968122fa None admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:11.117985 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3641bb1-a457-4583-b5e1-87e2968122fa None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:55:11.118295 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3641bb1-a457-4583-b5e1-87e2968122fa None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:11.119726 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3641bb1-a457-4583-b5e1-87e2968122fa None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:11.121016 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4141ee0d-9224-4703-bc16-537ff772802b None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:11.138922 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 267/528] 10.222.0.46 () {56 vars in 1139 bytes} [Thu May 26 14:55:10 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:55:11.139660 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3641bb1-a457-4583-b5e1-87e2968122fa None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:11.163311 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fca2f9-70c0-4323-9ce4-242d0e9bc69e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:11.163971 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fca2f9-70c0-4323-9ce4-242d0e9bc69e None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:11.164254 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fca2f9-70c0-4323-9ce4-242d0e9bc69e None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:11.164534 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fca2f9-70c0-4323-9ce4-242d0e9bc69e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:11.165362 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fca2f9-70c0-4323-9ce4-242d0e9bc69e None admin] RBAC: Adding query filter params (role.id=eb76213820fa4c4e9b0a5d5ac2a08b41, scope.project.id=a17fc8227d4c4da6b2847b350e912ee3, user.id=0a436dbdee624f44ad76cca8416b0ad1) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:55:11.165706 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fca2f9-70c0-4323-9ce4-242d0e9bc69e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:11.167174 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fca2f9-70c0-4323-9ce4-242d0e9bc69e None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:11.181816 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 262/529] 10.222.0.46 () {56 vars in 1374 bytes} [Thu May 26 14:55:11 2022] GET /identity/v3/role_assignments?user.id=0a436dbdee624f44ad76cca8416b0ad1&scope.project.id=a17fc8227d4c4da6b2847b350e912ee3&role.id=eb76213820fa4c4e9b0a5d5ac2a08b41 => generated 259 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:55:11.181816 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fca2f9-70c0-4323-9ce4-242d0e9bc69e None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:11.192910 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:11.302152 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fca2f9-70c0-4323-9ce4-242d0e9bc69e None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:11.306449 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3641bb1-a457-4583-b5e1-87e2968122fa None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:11.344517 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d738fd06-713a-446a-8f07-5cd6599fd09c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:11.345208 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d738fd06-713a-446a-8f07-5cd6599fd09c None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:11.345535 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d738fd06-713a-446a-8f07-5cd6599fd09c None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:11.345818 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d738fd06-713a-446a-8f07-5cd6599fd09c None admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:11.346559 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d738fd06-713a-446a-8f07-5cd6599fd09c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:55:11.346869 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d738fd06-713a-446a-8f07-5cd6599fd09c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:11.348287 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d738fd06-713a-446a-8f07-5cd6599fd09c None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:11.357736 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21771db7-8eda-48b6-b43f-1c1810fe7c90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:11.359304 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21771db7-8eda-48b6-b43f-1c1810fe7c90 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:11.359715 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21771db7-8eda-48b6-b43f-1c1810fe7c90 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:11.360391 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21771db7-8eda-48b6-b43f-1c1810fe7c90 None admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:11.361359 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21771db7-8eda-48b6-b43f-1c1810fe7c90 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:55:11.361808 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21771db7-8eda-48b6-b43f-1c1810fe7c90 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:11.363959 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 263/530] 10.222.0.46 () {56 vars in 1109 bytes} [Thu May 26 14:55:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:55:11.363959 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d738fd06-713a-446a-8f07-5cd6599fd09c None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:11.367170 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21771db7-8eda-48b6-b43f-1c1810fe7c90 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:11.405792 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 268/531] 10.222.0.46 () {56 vars in 1148 bytes} [Thu May 26 14:55:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:55:11.406943 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21771db7-8eda-48b6-b43f-1c1810fe7c90 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:11.432533 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:11.454372 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:11.489827 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d738fd06-713a-446a-8f07-5cd6599fd09c None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:11.527617 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb29f604-9b83-437f-8177-802c0ef9d0b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:11.528258 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb29f604-9b83-437f-8177-802c0ef9d0b2 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:11.528538 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb29f604-9b83-437f-8177-802c0ef9d0b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:11.528812 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb29f604-9b83-437f-8177-802c0ef9d0b2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:11.531296 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21771db7-8eda-48b6-b43f-1c1810fe7c90 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:11.575372 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb04b56-9a69-4567-9341-b1d4aca2151a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:11.576448 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb04b56-9a69-4567-9341-b1d4aca2151a None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:11.576990 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb04b56-9a69-4567-9341-b1d4aca2151a None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:11.577415 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb04b56-9a69-4567-9341-b1d4aca2151a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:11.585544 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb04b56-9a69-4567-9341-b1d4aca2151a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:11.587335 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb04b56-9a69-4567-9341-b1d4aca2151a None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:11.602976 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-fdb04b56-9a69-4567-9341-b1d4aca2151a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 14:55:11.602976 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:55:11.602976 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:55:11.602976 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:55:11.602976 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:55:11.602976 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:55:11.602976 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:55:11.602976 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:55:11.602976 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:55:11.602976 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:55:11.602976 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:55:11.602976 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:55:11.602976 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 14:55:11.602976 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 14:55:11.602976 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 14:55:11.604063 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 14:55:11.604063 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:55:11.604063 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:55:11.604063 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:55:11.604063 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:55:11.604063 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:55:11.604063 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:55:11.604063 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:55:11.604063 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 26 14:55:11.604063 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:55:11.604063 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 26 14:55:11.604063 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:55:11.604063 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:55:11.604063 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:55:11.604063 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 26 14:55:11.604063 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:55:11.604063 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 26 14:55:11.605130 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 14:55:11.605130 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 14:55:11.605130 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 14:55:11.605130 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 14:55:11.605130 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 14:55:11.605130 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 14:55:11.605130 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 14:55:11.605130 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 26 14:55:11.607143 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 269/532] 10.222.0.46 () {56 vars in 1111 bytes} [Thu May 26 14:55:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:55:11.607673 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb29f604-9b83-437f-8177-802c0ef9d0b2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:11.608250 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb04b56-9a69-4567-9341-b1d4aca2151a None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:11.609172 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb29f604-9b83-437f-8177-802c0ef9d0b2 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:11.690807 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:11.715749 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:11.739401 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 264/533] 10.222.0.46 () {58 vars in 1282 bytes} [Thu May 26 14:55:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 14:55:11.740324 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb29f604-9b83-437f-8177-802c0ef9d0b2 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:11.741653 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb04b56-9a69-4567-9341-b1d4aca2151a None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:11.781202 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ccbb65-4629-46e5-990b-ec516518fbbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:11.782713 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ccbb65-4629-46e5-990b-ec516518fbbd None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:11.783206 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ccbb65-4629-46e5-990b-ec516518fbbd None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:11.783633 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ccbb65-4629-46e5-990b-ec516518fbbd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:11.784669 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ccbb65-4629-46e5-990b-ec516518fbbd None admin] RBAC: Adding query filter params (role.id=eb76213820fa4c4e9b0a5d5ac2a08b41, scope.project.id=a17fc8227d4c4da6b2847b350e912ee3, user.id=d6639353c70449d1b3d10cd6487c3b00) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:55:11.785264 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ccbb65-4629-46e5-990b-ec516518fbbd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:11.787969 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ccbb65-4629-46e5-990b-ec516518fbbd None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:11.795391 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 270/534] 10.222.0.46 () {56 vars in 1374 bytes} [Thu May 26 14:55:11 2022] GET /identity/v3/role_assignments?user.id=d6639353c70449d1b3d10cd6487c3b00&scope.project.id=a17fc8227d4c4da6b2847b350e912ee3&role.id=eb76213820fa4c4e9b0a5d5ac2a08b41 => generated 259 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:55:11.796780 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ccbb65-4629-46e5-990b-ec516518fbbd None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:11.875563 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb29f604-9b83-437f-8177-802c0ef9d0b2 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:11.922492 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61635733-78d5-4131-8d9d-fd46dffec904 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:11.923256 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61635733-78d5-4131-8d9d-fd46dffec904 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:11.923610 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61635733-78d5-4131-8d9d-fd46dffec904 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:11.923954 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61635733-78d5-4131-8d9d-fd46dffec904 None admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:11.924825 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61635733-78d5-4131-8d9d-fd46dffec904 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:55:11.925203 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61635733-78d5-4131-8d9d-fd46dffec904 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:11.926790 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61635733-78d5-4131-8d9d-fd46dffec904 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:11.942327 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 265/535] 10.222.0.46 () {56 vars in 1148 bytes} [Thu May 26 14:55:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:55:11.948529 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61635733-78d5-4131-8d9d-fd46dffec904 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:11.984785 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:12.040210 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ccbb65-4629-46e5-990b-ec516518fbbd None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:12.077150 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61635733-78d5-4131-8d9d-fd46dffec904 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:12.105325 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9eb3eb-4557-42e9-a7c7-24abbd646f01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:12.106500 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9eb3eb-4557-42e9-a7c7-24abbd646f01 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:12.106973 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9eb3eb-4557-42e9-a7c7-24abbd646f01 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:12.107485 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9eb3eb-4557-42e9-a7c7-24abbd646f01 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:12.125179 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:12.129361 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9eb3eb-4557-42e9-a7c7-24abbd646f01 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:12.131470 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9eb3eb-4557-42e9-a7c7-24abbd646f01 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:12.158103 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14de7e9f-1044-4904-8071-2782a26e5776 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:12.159137 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14de7e9f-1044-4904-8071-2782a26e5776 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:12.159614 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14de7e9f-1044-4904-8071-2782a26e5776 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:12.160076 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14de7e9f-1044-4904-8071-2782a26e5776 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:12.161182 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14de7e9f-1044-4904-8071-2782a26e5776 None admin] RBAC: Adding query filter params (role.id=eb76213820fa4c4e9b0a5d5ac2a08b41, scope.project.id=a17fc8227d4c4da6b2847b350e912ee3, user.id=d5b5b35614a547d783e6e77ddb8c1c03) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:55:12.161748 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14de7e9f-1044-4904-8071-2782a26e5776 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:12.163369 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14de7e9f-1044-4904-8071-2782a26e5776 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:12.172228 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-9c9eb3eb-4557-42e9-a7c7-24abbd646f01 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 14:55:12.172228 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:55:12.172228 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:55:12.172228 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:55:12.172228 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:55:12.172228 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:55:12.172228 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:55:12.172228 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:55:12.172228 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:55:12.172228 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:55:12.172228 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:55:12.172228 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:55:12.172228 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 14:55:12.172228 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 14:55:12.172228 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 14:55:12.173147 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 14:55:12.173147 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:55:12.173147 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:55:12.173147 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:55:12.173147 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:55:12.173147 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:55:12.173147 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:55:12.173147 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:55:12.173147 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 26 14:55:12.173147 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:55:12.173147 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 26 14:55:12.173147 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:55:12.173147 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:55:12.173147 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:55:12.173147 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 26 14:55:12.173147 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:55:12.173147 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 26 14:55:12.174003 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 14:55:12.174003 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 14:55:12.174003 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 14:55:12.174003 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 26 14:55:12.176449 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 271/536] 10.222.0.46 () {56 vars in 1108 bytes} [Thu May 26 14:55:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 380 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:55:12.178164 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 266/537] 10.222.0.46 () {56 vars in 1374 bytes} [Thu May 26 14:55:11 2022] GET /identity/v3/role_assignments?user.id=d5b5b35614a547d783e6e77ddb8c1c03&scope.project.id=a17fc8227d4c4da6b2847b350e912ee3&role.id=eb76213820fa4c4e9b0a5d5ac2a08b41 => generated 259 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:55:12.184352 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9eb3eb-4557-42e9-a7c7-24abbd646f01 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:12.322386 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9eb3eb-4557-42e9-a7c7-24abbd646f01 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:12.377010 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ec4b60-4da5-4545-b8a0-66b84e940b9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:12.377721 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ec4b60-4da5-4545-b8a0-66b84e940b9b None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:12.378007 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ec4b60-4da5-4545-b8a0-66b84e940b9b None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:12.378282 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ec4b60-4da5-4545-b8a0-66b84e940b9b None admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:12.379015 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ec4b60-4da5-4545-b8a0-66b84e940b9b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:55:12.379327 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ec4b60-4da5-4545-b8a0-66b84e940b9b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:12.380976 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ec4b60-4da5-4545-b8a0-66b84e940b9b None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:12.399615 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 272/538] 10.222.0.46 () {56 vars in 1109 bytes} [Thu May 26 14:55:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:55:12.405147 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14de7e9f-1044-4904-8071-2782a26e5776 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:12.519075 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:12.527697 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14de7e9f-1044-4904-8071-2782a26e5776 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:12.565524 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce19536-6225-472f-9b7e-a60ff5f59323 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:12.566361 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce19536-6225-472f-9b7e-a60ff5f59323 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:12.566622 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce19536-6225-472f-9b7e-a60ff5f59323 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:12.566901 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce19536-6225-472f-9b7e-a60ff5f59323 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:12.575684 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce19536-6225-472f-9b7e-a60ff5f59323 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:12.577110 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce19536-6225-472f-9b7e-a60ff5f59323 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:12.596751 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-0ce19536-6225-472f-9b7e-a60ff5f59323 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 14:55:12.596751 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:55:12.596751 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:55:12.596751 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:55:12.596751 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:55:12.596751 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:55:12.596751 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:55:12.596751 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:55:12.596751 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:55:12.596751 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:55:12.596751 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:55:12.596751 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:55:12.596751 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 14:55:12.596751 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 14:55:12.596751 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 14:55:12.597832 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 14:55:12.597832 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:55:12.597832 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:55:12.597832 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:55:12.597832 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:55:12.597832 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:55:12.597832 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:55:12.597832 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:55:12.597832 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 26 14:55:12.597832 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:55:12.597832 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 26 14:55:12.597832 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:55:12.597832 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:55:12.597832 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:55:12.597832 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 26 14:55:12.597832 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:55:12.597832 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 26 14:55:12.598847 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 14:55:12.598847 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 14:55:12.598847 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 14:55:12.598847 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 14:55:12.598847 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 14:55:12.598847 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 14:55:12.598847 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 14:55:12.598847 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 26 14:55:12.601118 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 267/539] 10.222.0.46 () {56 vars in 1111 bytes} [Thu May 26 14:55:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:55:12.606961 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ec4b60-4da5-4545-b8a0-66b84e940b9b None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:12.708435 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:12.783644 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ec4b60-4da5-4545-b8a0-66b84e940b9b None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:12.825676 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80b9cb4-3432-44e6-b70a-fabfc9dbe334 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:12.826399 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80b9cb4-3432-44e6-b70a-fabfc9dbe334 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:12.826622 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80b9cb4-3432-44e6-b70a-fabfc9dbe334 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:12.826897 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80b9cb4-3432-44e6-b70a-fabfc9dbe334 None admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:12.827746 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80b9cb4-3432-44e6-b70a-fabfc9dbe334 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:55:12.828054 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80b9cb4-3432-44e6-b70a-fabfc9dbe334 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:12.829570 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e80b9cb4-3432-44e6-b70a-fabfc9dbe334 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:12.846594 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 273/540] 10.222.0.46 () {56 vars in 1148 bytes} [Thu May 26 14:55:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:55:12.852498 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce19536-6225-472f-9b7e-a60ff5f59323 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:12.910500 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a14d226-48dd-4438-a12a-b228a6c2d270 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:12.910820 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a14d226-48dd-4438-a12a-b228a6c2d270 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:12.911058 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a14d226-48dd-4438-a12a-b228a6c2d270 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:12.912413 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 274/541] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 14:55:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 14:55:12.924748 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b93cd3-e585-4d94-b2c5-f3b323b26154 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:12.925186 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b93cd3-e585-4d94-b2c5-f3b323b26154 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:12.925512 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b93cd3-e585-4d94-b2c5-f3b323b26154 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:12.958693 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-d9b93cd3-e585-4d94-b2c5-f3b323b26154 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:55:12.959969 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-d9b93cd3-e585-4d94-b2c5-f3b323b26154 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:55:12.978289 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce19536-6225-472f-9b7e-a60ff5f59323 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:13.048884 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a209175-c0b6-4d28-8437-56539f2e6527 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:13.049887 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a209175-c0b6-4d28-8437-56539f2e6527 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:13.050221 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a209175-c0b6-4d28-8437-56539f2e6527 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:13.050501 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a209175-c0b6-4d28-8437-56539f2e6527 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:13.051459 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a209175-c0b6-4d28-8437-56539f2e6527 None admin] RBAC: Adding query filter params (role.id=eb76213820fa4c4e9b0a5d5ac2a08b41, scope.project.id=a17fc8227d4c4da6b2847b350e912ee3, user.id=0156a2f6f3794f479e76b85bdbbe5c26) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:55:13.053625 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a209175-c0b6-4d28-8437-56539f2e6527 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:13.055141 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a209175-c0b6-4d28-8437-56539f2e6527 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:13.071073 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 268/542] 10.222.0.46 () {56 vars in 1374 bytes} [Thu May 26 14:55:12 2022] GET /identity/v3/role_assignments?user.id=0156a2f6f3794f479e76b85bdbbe5c26&scope.project.id=a17fc8227d4c4da6b2847b350e912ee3&role.id=eb76213820fa4c4e9b0a5d5ac2a08b41 => generated 259 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:55:13.111883 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 275/543] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 14:55:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 14:55:13.136861 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a209175-c0b6-4d28-8437-56539f2e6527 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:13.169703 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:13.285669 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a209175-c0b6-4d28-8437-56539f2e6527 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:13.342674 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7dfc0f-0465-4ab4-b877-d4ee5c9084fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:13.343852 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7dfc0f-0465-4ab4-b877-d4ee5c9084fb None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:13.344363 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7dfc0f-0465-4ab4-b877-d4ee5c9084fb None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:13.344858 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7dfc0f-0465-4ab4-b877-d4ee5c9084fb None admin] PATH_INFO: `/v3/roles/service` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:13.365181 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7dfc0f-0465-4ab4-b877-d4ee5c9084fb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 14:55:13.374056 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-6e7dfc0f-0465-4ab4-b877-d4ee5c9084fb None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 26 14:55:13.374056 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:55:13.374056 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:55:13.374056 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:55:13.374056 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:55:13.374056 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:55:13.374056 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:55:13.374056 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:55:13.374056 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:55:13.374056 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:55:13.374056 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:55:13.374056 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:55:13.374056 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 14:55:13.374056 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 14:55:13.374056 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 14:55:13.374056 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 14:55:13.374967 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 14:55:13.374967 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 14:55:13.374967 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 14:55:13.374967 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 14:55:13.374967 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:55:13.374967 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:55:13.374967 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:55:13.374967 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:55:13.374967 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:55:13.374967 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:55:13.374967 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:55:13.374967 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 26 14:55:13.374967 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:55:13.374967 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 26 14:55:13.374967 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:55:13.374967 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:55:13.375766 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:55:13.375766 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 26 14:55:13.375766 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:55:13.375766 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 26 14:55:13.375766 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 14:55:13.375766 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 14:55:13.375766 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 14:55:13.375766 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 14:55:13.375766 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 14:55:13.375766 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 14:55:13.375766 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 26 14:55:13.375766 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 26 14:55:13.378741 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 269/544] 10.222.0.46 () {56 vars in 1102 bytes} [Thu May 26 14:55:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:55:13.386615 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b93cd3-e585-4d94-b2c5-f3b323b26154 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:13.448895 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:13.520386 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b93cd3-e585-4d94-b2c5-f3b323b26154 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:13.556435 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4253a4f6-26de-4c6f-8a6f-88388c4aa613 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:13.557515 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4253a4f6-26de-4c6f-8a6f-88388c4aa613 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:13.558617 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4253a4f6-26de-4c6f-8a6f-88388c4aa613 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:13.559144 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4253a4f6-26de-4c6f-8a6f-88388c4aa613 None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:13.560098 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4253a4f6-26de-4c6f-8a6f-88388c4aa613 None admin] RBAC: Adding query filter params (name=service) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:55:13.560602 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4253a4f6-26de-4c6f-8a6f-88388c4aa613 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:13.562224 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4253a4f6-26de-4c6f-8a6f-88388c4aa613 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:13.571536 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 276/545] 10.222.0.46 () {56 vars in 1103 bytes} [Thu May 26 14:55:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:55:13.582562 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7dfc0f-0465-4ab4-b877-d4ee5c9084fb None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:13.678805 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7622399-c981-4cb0-addc-9b38a694623e None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:13.679442 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7622399-c981-4cb0-addc-9b38a694623e None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:13.679928 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7622399-c981-4cb0-addc-9b38a694623e None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:13.681878 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 277/546] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 14:55:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 14:55:13.689689 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c351b5d8-47de-435f-8282-0b91511732c9 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:13.690246 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c351b5d8-47de-435f-8282-0b91511732c9 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:13.690727 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c351b5d8-47de-435f-8282-0b91511732c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:13.709914 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-c351b5d8-47de-435f-8282-0b91511732c9 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:55:13.712338 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-c351b5d8-47de-435f-8282-0b91511732c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:55:13.728165 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:13.799443 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7dfc0f-0465-4ab4-b877-d4ee5c9084fb None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:13.799838 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 278/547] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 14:55:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 14:55:13.822539 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c351b5d8-47de-435f-8282-0b91511732c9 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:13.893933 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:13.907211 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edce692-ceed-4adf-b95a-a44b4a6ad3cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:13.908200 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edce692-ceed-4adf-b95a-a44b4a6ad3cb None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:13.908310 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edce692-ceed-4adf-b95a-a44b4a6ad3cb None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:13.908567 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edce692-ceed-4adf-b95a-a44b4a6ad3cb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:13.936449 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c351b5d8-47de-435f-8282-0b91511732c9 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:13.946367 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3edce692-ceed-4adf-b95a-a44b4a6ad3cb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:13.947888 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3edce692-ceed-4adf-b95a-a44b4a6ad3cb None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:13.976046 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703329fc-ae44-4d1b-86b5-f595cbb5747c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:13.976244 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-3edce692-ceed-4adf-b95a-a44b4a6ad3cb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 14:55:13.976244 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:55:13.976244 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:55:13.976244 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:55:13.976244 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:55:13.976244 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:55:13.976244 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:55:13.976244 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:55:13.976244 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:55:13.976244 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:55:13.976244 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:55:13.976244 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:55:13.976244 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 14:55:13.976244 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 14:55:13.976244 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 14:55:13.977128 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 14:55:13.977128 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:55:13.977128 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:55:13.977128 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:55:13.977128 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:55:13.977128 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:55:13.977128 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:55:13.977128 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:55:13.977128 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 26 14:55:13.977128 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:55:13.977128 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 26 14:55:13.977128 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:55:13.977128 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:55:13.977128 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:55:13.977128 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 26 14:55:13.977128 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:55:13.977128 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 26 14:55:13.979080 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 14:55:13.979080 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 14:55:13.979080 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 14:55:13.979080 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 26 14:55:13.979080 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 270/548] 10.222.0.46 () {56 vars in 1108 bytes} [Thu May 26 14:55:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 395 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:55:13.980893 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703329fc-ae44-4d1b-86b5-f595cbb5747c None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:13.981393 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703329fc-ae44-4d1b-86b5-f595cbb5747c None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:13.981903 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703329fc-ae44-4d1b-86b5-f595cbb5747c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:13.985209 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edce692-ceed-4adf-b95a-a44b4a6ad3cb None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:13.994787 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703329fc-ae44-4d1b-86b5-f595cbb5747c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 14:55:14.003966 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-703329fc-ae44-4d1b-86b5-f595cbb5747c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 26 14:55:14.003966 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:55:14.003966 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:55:14.003966 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:55:14.003966 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:55:14.003966 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:55:14.003966 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:55:14.003966 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:55:14.003966 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:55:14.003966 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:55:14.003966 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:55:14.003966 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:55:14.003966 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 14:55:14.003966 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 14:55:14.003966 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 14:55:14.003966 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 14:55:14.006219 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 14:55:14.006219 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 14:55:14.006219 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 14:55:14.006219 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 14:55:14.006219 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:55:14.006219 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:55:14.006219 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:55:14.006219 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:55:14.006219 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:55:14.006219 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:55:14.006219 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:55:14.006219 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 26 14:55:14.006219 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:55:14.006219 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 26 14:55:14.006219 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:55:14.006219 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:55:14.007219 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:55:14.007219 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 26 14:55:14.007219 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:55:14.007219 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 26 14:55:14.007219 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 14:55:14.007219 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 14:55:14.007219 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 14:55:14.007219 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 14:55:14.007219 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 14:55:14.007219 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 14:55:14.007219 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 26 14:55:14.007219 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 26 14:55:14.014913 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 279/549] 10.222.0.46 () {56 vars in 1102 bytes} [Thu May 26 14:55:13 2022] GET /identity/v3/roles/service => generated 85 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:55:14.016247 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703329fc-ae44-4d1b-86b5-f595cbb5747c None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:14.129251 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:14.138421 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703329fc-ae44-4d1b-86b5-f595cbb5747c None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:14.180901 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c943b8b9-bb43-4091-92d3-189a279b71c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:14.181985 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c943b8b9-bb43-4091-92d3-189a279b71c2 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:14.182942 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c943b8b9-bb43-4091-92d3-189a279b71c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:14.183443 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c943b8b9-bb43-4091-92d3-189a279b71c2 None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:14.184643 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c943b8b9-bb43-4091-92d3-189a279b71c2 None admin] RBAC: Adding query filter params (name=service) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:55:14.185165 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c943b8b9-bb43-4091-92d3-189a279b71c2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:14.186815 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c943b8b9-bb43-4091-92d3-189a279b71c2 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:14.201689 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 280/550] 10.222.0.46 () {56 vars in 1103 bytes} [Thu May 26 14:55:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 14:55:14.204317 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c943b8b9-bb43-4091-92d3-189a279b71c2 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:14.227908 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edce692-ceed-4adf-b95a-a44b4a6ad3cb None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:14.278968 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:14.288823 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2158e3-57f9-4a4c-9241-8de1c8996124 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:14.289521 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2158e3-57f9-4a4c-9241-8de1c8996124 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:14.289808 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2158e3-57f9-4a4c-9241-8de1c8996124 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:14.290084 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2158e3-57f9-4a4c-9241-8de1c8996124 None admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:14.290922 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2158e3-57f9-4a4c-9241-8de1c8996124 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:55:14.291243 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2158e3-57f9-4a4c-9241-8de1c8996124 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:14.292820 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2158e3-57f9-4a4c-9241-8de1c8996124 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:14.310046 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 271/551] 10.222.0.46 () {56 vars in 1109 bytes} [Thu May 26 14:55:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 14:55:14.311447 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2158e3-57f9-4a4c-9241-8de1c8996124 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:14.345735 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c943b8b9-bb43-4091-92d3-189a279b71c2 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:14.383852 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5cd0678-d8c8-47f9-8ce1-11433e7eed8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:14.384692 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cd0678-d8c8-47f9-8ce1-11433e7eed8b None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:14.385043 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cd0678-d8c8-47f9-8ce1-11433e7eed8b None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:14.385392 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5cd0678-d8c8-47f9-8ce1-11433e7eed8b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:14.398215 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cd0678-d8c8-47f9-8ce1-11433e7eed8b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:14.399898 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5cd0678-d8c8-47f9-8ce1-11433e7eed8b None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:14.419643 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-b5cd0678-d8c8-47f9-8ce1-11433e7eed8b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 14:55:14.419643 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:55:14.419643 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:55:14.419643 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:55:14.419643 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:55:14.419643 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:55:14.419643 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:55:14.419643 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:55:14.419643 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:55:14.419643 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:55:14.419643 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:55:14.419643 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:55:14.419643 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 14:55:14.419643 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 14:55:14.419643 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 14:55:14.420711 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 14:55:14.420711 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:55:14.420711 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:55:14.420711 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:55:14.420711 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:55:14.420711 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:55:14.420711 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:55:14.420711 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:55:14.420711 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 26 14:55:14.420711 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:55:14.420711 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 26 14:55:14.420711 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:55:14.420711 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:55:14.420711 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:55:14.420711 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 26 14:55:14.420711 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:55:14.420711 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 26 14:55:14.421822 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 14:55:14.421822 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 14:55:14.421822 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 14:55:14.421822 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 26 14:55:14.421822 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 281/552] 10.222.0.46 () {56 vars in 1108 bytes} [Thu May 26 14:55:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 14:55:14.427493 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a658a417-69ed-4307-844d-3ed7d5fe2c4b None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:14.427899 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a658a417-69ed-4307-844d-3ed7d5fe2c4b None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:14.428527 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a658a417-69ed-4307-844d-3ed7d5fe2c4b None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:14.434158 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 282/553] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 14:55:14 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 14:55:14.434158 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a658a417-69ed-4307-844d-3ed7d5fe2c4b None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:14.439596 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2158e3-57f9-4a4c-9241-8de1c8996124 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:14.444843 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:14.478662 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08abbaad-8107-4e88-8d10-9b9662e239fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:14.479825 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08abbaad-8107-4e88-8d10-9b9662e239fd None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:14.480260 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08abbaad-8107-4e88-8d10-9b9662e239fd None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:14.480752 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08abbaad-8107-4e88-8d10-9b9662e239fd None admin] PATH_INFO: `/v3/users/swift` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:14.488597 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08abbaad-8107-4e88-8d10-9b9662e239fd None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:14.490357 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08abbaad-8107-4e88-8d10-9b9662e239fd None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:14.507259 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-08abbaad-8107-4e88-8d10-9b9662e239fd None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 26 14:55:14.507259 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:55:14.507259 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:55:14.507259 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:55:14.507259 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:55:14.507259 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:55:14.507259 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:55:14.507259 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:55:14.507259 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:55:14.507259 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:55:14.507259 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:55:14.507259 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:55:14.507259 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 14:55:14.507259 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 14:55:14.507259 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 14:55:14.507259 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 14:55:14.508248 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:55:14.508248 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:55:14.508248 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 14:55:14.508248 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 14:55:14.508248 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 14:55:14.508248 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 14:55:14.508248 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:55:14.508248 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:55:14.508248 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:55:14.508248 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:55:14.508248 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:55:14.508248 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 26 14:55:14.508248 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:55:14.508248 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 26 14:55:14.508248 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:55:14.508248 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:55:14.508248 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:55:14.508248 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 26 14:55:14.509235 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:55:14.509235 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 26 14:55:14.509235 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 14:55:14.509235 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 14:55:14.509235 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 14:55:14.509235 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 14:55:14.509235 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 14:55:14.509235 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 14:55:14.509235 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 26 14:55:14.509235 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 26 14:55:14.512287 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 272/554] 10.222.0.46 () {56 vars in 1096 bytes} [Thu May 26 14:55:14 2022] GET /identity/v3/users/swift => generated 83 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:55:14.513854 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d57c61e-92c6-4765-b64e-5e7f8fd3ddab None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:14.514276 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d57c61e-92c6-4765-b64e-5e7f8fd3ddab None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:14.514699 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d57c61e-92c6-4765-b64e-5e7f8fd3ddab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:14.533679 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-9d57c61e-92c6-4765-b64e-5e7f8fd3ddab None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:55:14.535126 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-9d57c61e-92c6-4765-b64e-5e7f8fd3ddab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:55:14.556317 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a658a417-69ed-4307-844d-3ed7d5fe2c4b None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:14.583785 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:14.603613 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f6978f-92ac-4e2b-8515-04b314e7a804 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:14.611122 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f6978f-92ac-4e2b-8515-04b314e7a804 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:14.611637 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f6978f-92ac-4e2b-8515-04b314e7a804 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:14.612082 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6f6978f-92ac-4e2b-8515-04b314e7a804 None admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:14.613646 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f6978f-92ac-4e2b-8515-04b314e7a804 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:55:14.614120 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f6978f-92ac-4e2b-8515-04b314e7a804 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:14.615680 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6f6978f-92ac-4e2b-8515-04b314e7a804 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:14.622128 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 273/555] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 14:55:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 14:55:14.622128 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d57c61e-92c6-4765-b64e-5e7f8fd3ddab None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:14.646245 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 283/556] 10.222.0.46 () {56 vars in 1109 bytes} [Thu May 26 14:55:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 14:55:14.654639 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f6978f-92ac-4e2b-8515-04b314e7a804 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:14.742409 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:14.761616 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d57c61e-92c6-4765-b64e-5e7f8fd3ddab None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:14.782369 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6f6978f-92ac-4e2b-8515-04b314e7a804 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:14.802025 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bf16f1-f7d5-4403-8985-98664fa3aea7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:14.803100 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bf16f1-f7d5-4403-8985-98664fa3aea7 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:14.803623 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bf16f1-f7d5-4403-8985-98664fa3aea7 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:14.804095 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bf16f1-f7d5-4403-8985-98664fa3aea7 None admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:14.805257 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45bf16f1-f7d5-4403-8985-98664fa3aea7 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:55:14.805951 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45bf16f1-f7d5-4403-8985-98664fa3aea7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:14.808277 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45bf16f1-f7d5-4403-8985-98664fa3aea7 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:14.820517 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc16907-0c28-4006-a8e7-e935ea412fab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:14.821322 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc16907-0c28-4006-a8e7-e935ea412fab None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:14.821642 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc16907-0c28-4006-a8e7-e935ea412fab None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:14.821922 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc16907-0c28-4006-a8e7-e935ea412fab None admin] PATH_INFO: `/v3/users/glance` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:14.834260 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc16907-0c28-4006-a8e7-e935ea412fab None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:14.834260 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc16907-0c28-4006-a8e7-e935ea412fab None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:14.834770 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 274/557] 10.222.0.46 () {56 vars in 1135 bytes} [Thu May 26 14:55:14 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:55:14.835947 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bf16f1-f7d5-4403-8985-98664fa3aea7 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:14.848149 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-8cc16907-0c28-4006-a8e7-e935ea412fab None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 26 14:55:14.848149 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:55:14.848149 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:55:14.848149 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:55:14.848149 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:55:14.848149 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:55:14.848149 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:55:14.848149 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:55:14.848149 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:55:14.848149 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:55:14.848149 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:55:14.848149 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:55:14.848149 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 14:55:14.848149 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 14:55:14.848149 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 14:55:14.848149 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 14:55:14.849782 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:55:14.849782 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:55:14.849782 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 14:55:14.849782 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 14:55:14.849782 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 14:55:14.849782 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 14:55:14.849782 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:55:14.849782 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:55:14.849782 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:55:14.849782 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:55:14.849782 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:55:14.849782 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 26 14:55:14.849782 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:55:14.849782 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 26 14:55:14.849782 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:55:14.849782 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:55:14.849782 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:55:14.849782 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 26 14:55:14.851296 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:55:14.851296 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 26 14:55:14.851296 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 14:55:14.851296 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 14:55:14.851296 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 14:55:14.851296 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 14:55:14.851296 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 14:55:14.851296 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 14:55:14.851296 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 26 14:55:14.851296 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 26 14:55:14.851296 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 284/558] 10.222.0.46 () {56 vars in 1099 bytes} [Thu May 26 14:55:14 2022] GET /identity/v3/users/glance => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:55:14.851296 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc16907-0c28-4006-a8e7-e935ea412fab None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:14.880436 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:14.955041 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bf16f1-f7d5-4403-8985-98664fa3aea7 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:14.968313 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc16907-0c28-4006-a8e7-e935ea412fab None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:14.994233 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e96ac3-a3a9-4b25-bf65-a681788949a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:14.994948 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e96ac3-a3a9-4b25-bf65-a681788949a2 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:14.995308 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e96ac3-a3a9-4b25-bf65-a681788949a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:14.996068 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e96ac3-a3a9-4b25-bf65-a681788949a2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:14.999475 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:15.009432 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e96ac3-a3a9-4b25-bf65-a681788949a2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 14:55:15.012006 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93bfdd2-c89d-40e1-b8bd-3405e9a0c32a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:15.012686 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93bfdd2-c89d-40e1-b8bd-3405e9a0c32a None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:15.012975 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93bfdd2-c89d-40e1-b8bd-3405e9a0c32a None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:15.013259 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93bfdd2-c89d-40e1-b8bd-3405e9a0c32a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:15.017582 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-b1e96ac3-a3a9-4b25-bf65-a681788949a2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 26 14:55:15.017582 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:55:15.017582 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:55:15.017582 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:55:15.017582 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:55:15.017582 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:55:15.017582 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:55:15.017582 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:55:15.017582 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:55:15.017582 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:55:15.017582 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:55:15.017582 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:55:15.017582 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 14:55:15.017582 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 14:55:15.017582 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 14:55:15.017582 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 14:55:15.018659 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 14:55:15.018659 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 14:55:15.018659 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 14:55:15.018659 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 14:55:15.018659 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:55:15.018659 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:55:15.018659 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:55:15.018659 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:55:15.018659 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:55:15.018659 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:55:15.018659 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:55:15.018659 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 26 14:55:15.018659 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:55:15.018659 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 26 14:55:15.018659 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:55:15.018659 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:55:15.019483 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:55:15.019483 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 26 14:55:15.019483 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:55:15.019483 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 26 14:55:15.019483 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 14:55:15.019483 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 14:55:15.019483 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 14:55:15.019483 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 14:55:15.019483 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 14:55:15.019483 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 14:55:15.019483 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 26 14:55:15.019483 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 26 14:55:15.019483 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 275/559] 10.222.0.46 () {56 vars in 1102 bytes} [Thu May 26 14:55:14 2022] GET /identity/v3/roles/service => generated 85 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:55:15.020157 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b5fb30-8b24-415f-8a55-4261a785b9c2 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:15.020735 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b5fb30-8b24-415f-8a55-4261a785b9c2 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:15.020735 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b5fb30-8b24-415f-8a55-4261a785b9c2 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:15.022014 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 276/560] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 14:55:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 14:55:15.026651 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b5fb30-8b24-415f-8a55-4261a785b9c2 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:15.108247 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93bfdd2-c89d-40e1-b8bd-3405e9a0c32a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:15.110661 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93bfdd2-c89d-40e1-b8bd-3405e9a0c32a None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:15.135475 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:15.153061 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b5fb30-8b24-415f-8a55-4261a785b9c2 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:15.191812 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a5e18d-70c5-4ef0-865c-bb1fca305263 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:15.192482 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a5e18d-70c5-4ef0-865c-bb1fca305263 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:15.192771 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a5e18d-70c5-4ef0-865c-bb1fca305263 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:15.193044 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a5e18d-70c5-4ef0-865c-bb1fca305263 None admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:15.193923 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a5e18d-70c5-4ef0-865c-bb1fca305263 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:55:15.194228 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a5e18d-70c5-4ef0-865c-bb1fca305263 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:15.195685 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a5e18d-70c5-4ef0-865c-bb1fca305263 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:15.213489 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 277/561] 10.222.0.46 () {56 vars in 1137 bytes} [Thu May 26 14:55:15 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:55:15.214730 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a5e18d-70c5-4ef0-865c-bb1fca305263 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:15.236688 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 285/562] 10.222.0.46 () {58 vars in 1282 bytes} [Thu May 26 14:55:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 14:55:15.236688 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9802ec-76e8-4332-915d-486f53d059fb None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:15.236688 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9802ec-76e8-4332-915d-486f53d059fb None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:15.236688 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d9802ec-76e8-4332-915d-486f53d059fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:15.252746 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-1d9802ec-76e8-4332-915d-486f53d059fb None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:55:15.254079 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-1d9802ec-76e8-4332-915d-486f53d059fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:55:15.278915 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:15.331360 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a5e18d-70c5-4ef0-865c-bb1fca305263 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:15.338482 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 286/563] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 14:55:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 14:55:15.340029 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9802ec-76e8-4332-915d-486f53d059fb None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:15.380635 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4568bc-3039-42d9-95e5-486c6459f74a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:15.381317 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4568bc-3039-42d9-95e5-486c6459f74a None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:15.381640 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4568bc-3039-42d9-95e5-486c6459f74a None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:15.381927 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4568bc-3039-42d9-95e5-486c6459f74a None admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:15.382685 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4568bc-3039-42d9-95e5-486c6459f74a None admin] RBAC: Adding query filter params (name=service) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:55:15.382995 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4568bc-3039-42d9-95e5-486c6459f74a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:15.384533 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4568bc-3039-42d9-95e5-486c6459f74a None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:15.397632 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 278/564] 10.222.0.46 () {56 vars in 1103 bytes} [Thu May 26 14:55:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:55:15.398908 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4568bc-3039-42d9-95e5-486c6459f74a None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:15.419702 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:15.476421 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d9802ec-76e8-4332-915d-486f53d059fb None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:15.521861 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7ee84e-a445-486b-a2e6-0f72f2e4ef3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:15.522898 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7ee84e-a445-486b-a2e6-0f72f2e4ef3e None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:15.523324 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7ee84e-a445-486b-a2e6-0f72f2e4ef3e None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:15.523739 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7ee84e-a445-486b-a2e6-0f72f2e4ef3e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:15.533607 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4568bc-3039-42d9-95e5-486c6459f74a None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:15.537030 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7ee84e-a445-486b-a2e6-0f72f2e4ef3e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:15.538736 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7ee84e-a445-486b-a2e6-0f72f2e4ef3e None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:15.560236 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-5d7ee84e-a445-486b-a2e6-0f72f2e4ef3e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 14:55:15.560236 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:55:15.560236 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:55:15.560236 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:55:15.560236 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:55:15.560236 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:55:15.560236 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:55:15.560236 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:55:15.560236 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:55:15.560236 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:55:15.560236 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:55:15.560236 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:55:15.560236 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 14:55:15.560236 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 14:55:15.560236 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 14:55:15.561212 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 14:55:15.561212 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:55:15.561212 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:55:15.561212 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:55:15.561212 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:55:15.561212 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:55:15.561212 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:55:15.561212 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:55:15.561212 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 26 14:55:15.561212 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:55:15.561212 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 26 14:55:15.561212 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:55:15.561212 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:55:15.561212 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:55:15.561212 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 26 14:55:15.561212 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:55:15.561212 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 26 14:55:15.562590 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 14:55:15.562590 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 14:55:15.562590 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 14:55:15.562590 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 26 14:55:15.562590 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 287/565] 10.222.0.46 () {56 vars in 1108 bytes} [Thu May 26 14:55:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:55:15.562590 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7ee84e-a445-486b-a2e6-0f72f2e4ef3e None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:15.576516 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:15.583229 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a84e5b-fb1b-41fa-a417-1f796686e900 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:15.584090 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a84e5b-fb1b-41fa-a417-1f796686e900 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:15.584362 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a84e5b-fb1b-41fa-a417-1f796686e900 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:15.584640 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a84e5b-fb1b-41fa-a417-1f796686e900 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:15.607257 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a84e5b-fb1b-41fa-a417-1f796686e900 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:15.610250 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a84e5b-fb1b-41fa-a417-1f796686e900 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:15.632580 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-f0a84e5b-fb1b-41fa-a417-1f796686e900 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 14:55:15.632580 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:55:15.632580 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:55:15.632580 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:55:15.632580 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:55:15.632580 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:55:15.632580 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:55:15.632580 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:55:15.632580 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:55:15.632580 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:55:15.632580 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:55:15.632580 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:55:15.632580 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 14:55:15.632580 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 14:55:15.632580 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 14:55:15.633526 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 14:55:15.633526 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:55:15.633526 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:55:15.633526 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:55:15.633526 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:55:15.633526 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:55:15.633526 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:55:15.633526 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:55:15.633526 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 26 14:55:15.633526 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:55:15.633526 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 26 14:55:15.633526 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:55:15.633526 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:55:15.633526 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:55:15.633526 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 26 14:55:15.633526 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:55:15.633526 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 26 14:55:15.640091 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 14:55:15.640091 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 14:55:15.640091 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 14:55:15.640091 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 26 14:55:15.640091 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 279/566] 10.222.0.46 () {56 vars in 1108 bytes} [Thu May 26 14:55:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:55:15.640091 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a84e5b-fb1b-41fa-a417-1f796686e900 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:15.717225 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7ee84e-a445-486b-a2e6-0f72f2e4ef3e None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:15.757818 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:15.783315 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126f1c7f-400c-40d0-b0f8-1ae16745104a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:15.784719 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126f1c7f-400c-40d0-b0f8-1ae16745104a None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:15.785284 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126f1c7f-400c-40d0-b0f8-1ae16745104a None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:15.785907 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126f1c7f-400c-40d0-b0f8-1ae16745104a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:15.805757 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126f1c7f-400c-40d0-b0f8-1ae16745104a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 14:55:15.806989 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a84e5b-fb1b-41fa-a417-1f796686e900 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:15.822002 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-126f1c7f-400c-40d0-b0f8-1ae16745104a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 26 14:55:15.822002 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:55:15.822002 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:55:15.822002 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:55:15.822002 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:55:15.822002 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:55:15.822002 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:55:15.822002 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:55:15.822002 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:55:15.822002 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:55:15.822002 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:55:15.822002 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:55:15.822002 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 14:55:15.822002 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 14:55:15.822002 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 14:55:15.822002 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 14:55:15.824251 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 14:55:15.824251 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 14:55:15.824251 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 14:55:15.824251 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 14:55:15.824251 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:55:15.824251 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:55:15.824251 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:55:15.824251 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:55:15.824251 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:55:15.824251 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:55:15.824251 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:55:15.824251 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 26 14:55:15.824251 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:55:15.824251 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 26 14:55:15.824251 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:55:15.824251 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:55:15.825918 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:55:15.825918 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 26 14:55:15.825918 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:55:15.825918 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 26 14:55:15.825918 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 14:55:15.825918 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 14:55:15.825918 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 14:55:15.825918 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 14:55:15.825918 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 14:55:15.825918 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 14:55:15.825918 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 26 14:55:15.825918 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 26 14:55:15.825918 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 288/567] 10.222.0.46 () {56 vars in 1102 bytes} [Thu May 26 14:55:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:55:15.828374 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126f1c7f-400c-40d0-b0f8-1ae16745104a None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:15.862260 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4e1df4-8e48-4faa-99ab-fc128f67fdaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:15.863642 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4e1df4-8e48-4faa-99ab-fc128f67fdaa None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:15.864102 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4e1df4-8e48-4faa-99ab-fc128f67fdaa None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:15.864590 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4e1df4-8e48-4faa-99ab-fc128f67fdaa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:15.889101 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4e1df4-8e48-4faa-99ab-fc128f67fdaa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:15.891883 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4e1df4-8e48-4faa-99ab-fc128f67fdaa None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:15.928161 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-bd4e1df4-8e48-4faa-99ab-fc128f67fdaa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 14:55:15.928161 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:55:15.928161 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:55:15.928161 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:55:15.928161 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:55:15.928161 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:55:15.928161 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:55:15.928161 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:55:15.928161 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:55:15.928161 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:55:15.928161 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:55:15.928161 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:55:15.928161 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 14:55:15.928161 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 14:55:15.928161 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 14:55:15.929428 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 14:55:15.929428 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:55:15.929428 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:55:15.929428 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:55:15.929428 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:55:15.929428 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:55:15.929428 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:55:15.929428 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:55:15.929428 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 26 14:55:15.929428 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:55:15.929428 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 26 14:55:15.929428 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:55:15.929428 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:55:15.929428 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:55:15.929428 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 26 14:55:15.929428 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:55:15.929428 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 26 14:55:15.930365 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 14:55:15.930365 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 14:55:15.930365 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 14:55:15.930365 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 26 14:55:15.930596 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 280/568] 10.222.0.46 () {56 vars in 1108 bytes} [Thu May 26 14:55:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:55:15.932268 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4e1df4-8e48-4faa-99ab-fc128f67fdaa None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:16.017960 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:16.083057 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126f1c7f-400c-40d0-b0f8-1ae16745104a None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:16.141401 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95abc847-0fe7-4f62-b4df-c6123c2c64c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:16.142168 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95abc847-0fe7-4f62-b4df-c6123c2c64c1 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:16.142463 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95abc847-0fe7-4f62-b4df-c6123c2c64c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:16.142743 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95abc847-0fe7-4f62-b4df-c6123c2c64c1 None admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:16.143839 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95abc847-0fe7-4f62-b4df-c6123c2c64c1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:55:16.144271 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95abc847-0fe7-4f62-b4df-c6123c2c64c1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:16.146459 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95abc847-0fe7-4f62-b4df-c6123c2c64c1 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:16.164447 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4e1df4-8e48-4faa-99ab-fc128f67fdaa None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:16.173781 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 289/569] 10.222.0.46 () {56 vars in 1109 bytes} [Thu May 26 14:55:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 348 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:55:16.174938 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95abc847-0fe7-4f62-b4df-c6123c2c64c1 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:16.197266 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:16.213059 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55e6f70-61fc-41ba-a60d-6ede96bbf581 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:16.214533 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55e6f70-61fc-41ba-a60d-6ede96bbf581 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:16.215016 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55e6f70-61fc-41ba-a60d-6ede96bbf581 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:16.215209 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55e6f70-61fc-41ba-a60d-6ede96bbf581 None admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:16.215997 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55e6f70-61fc-41ba-a60d-6ede96bbf581 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:55:16.216652 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55e6f70-61fc-41ba-a60d-6ede96bbf581 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:16.217980 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55e6f70-61fc-41ba-a60d-6ede96bbf581 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:16.233241 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 281/570] 10.222.0.46 () {56 vars in 1109 bytes} [Thu May 26 14:55:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:55:16.234920 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55e6f70-61fc-41ba-a60d-6ede96bbf581 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:16.305332 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95abc847-0fe7-4f62-b4df-c6123c2c64c1 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:16.339552 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:16.345712 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d765e7-827a-47a8-99fe-79b443d0521b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:16.346392 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d765e7-827a-47a8-99fe-79b443d0521b None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:16.346677 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d765e7-827a-47a8-99fe-79b443d0521b None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:16.346831 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55e6f70-61fc-41ba-a60d-6ede96bbf581 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:16.346937 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d765e7-827a-47a8-99fe-79b443d0521b None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:16.347694 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d765e7-827a-47a8-99fe-79b443d0521b None admin] RBAC: Adding query filter params (name=service) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:55:16.348004 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d765e7-827a-47a8-99fe-79b443d0521b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:16.349786 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d765e7-827a-47a8-99fe-79b443d0521b None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:16.358549 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 290/571] 10.222.0.46 () {56 vars in 1103 bytes} [Thu May 26 14:55:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:55:16.360074 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bc7acb-98b9-46ba-809a-86c360540bfc None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:16.360396 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bc7acb-98b9-46ba-809a-86c360540bfc None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:16.360646 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bc7acb-98b9-46ba-809a-86c360540bfc None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:16.363278 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 291/572] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 14:55:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 14:55:16.363278 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bc7acb-98b9-46ba-809a-86c360540bfc None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:16.387951 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64174cfd-8854-4376-805f-b5b1e3c262a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:16.390041 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64174cfd-8854-4376-805f-b5b1e3c262a8 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:16.391255 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64174cfd-8854-4376-805f-b5b1e3c262a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:16.391473 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64174cfd-8854-4376-805f-b5b1e3c262a8 None admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:16.392362 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64174cfd-8854-4376-805f-b5b1e3c262a8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:55:16.392640 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64174cfd-8854-4376-805f-b5b1e3c262a8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:16.396130 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64174cfd-8854-4376-805f-b5b1e3c262a8 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:16.409352 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 282/573] 10.222.0.46 () {56 vars in 1109 bytes} [Thu May 26 14:55:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:55:16.410252 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64174cfd-8854-4376-805f-b5b1e3c262a8 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:16.479927 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:16.507121 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bc7acb-98b9-46ba-809a-86c360540bfc None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:16.564850 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f287201-1c15-45a7-acb2-9b7694a0a7ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:16.566453 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f287201-1c15-45a7-acb2-9b7694a0a7ac None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:16.566951 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f287201-1c15-45a7-acb2-9b7694a0a7ac None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:16.567426 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f287201-1c15-45a7-acb2-9b7694a0a7ac None admin] PATH_INFO: `/v3/projects/service` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:16.577898 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f287201-1c15-45a7-acb2-9b7694a0a7ac None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:16.579676 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f287201-1c15-45a7-acb2-9b7694a0a7ac None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:16.594055 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64174cfd-8854-4376-805f-b5b1e3c262a8 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:16.598093 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-0f287201-1c15-45a7-acb2-9b7694a0a7ac None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 14:55:16.598093 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:55:16.598093 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:55:16.598093 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:55:16.598093 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:55:16.598093 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:55:16.598093 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:55:16.598093 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:55:16.598093 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:55:16.598093 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:55:16.598093 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:55:16.598093 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:55:16.598093 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 14:55:16.598093 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 14:55:16.598093 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 14:55:16.600027 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 14:55:16.600027 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:55:16.600027 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:55:16.600027 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:55:16.600027 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:55:16.600027 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:55:16.600027 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:55:16.600027 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:55:16.600027 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 26 14:55:16.600027 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:55:16.600027 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 26 14:55:16.600027 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:55:16.600027 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:55:16.600027 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:55:16.600027 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 26 14:55:16.600027 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:55:16.600027 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 26 14:55:16.601209 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 14:55:16.601209 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 14:55:16.601209 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 14:55:16.601209 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 14:55:16.601209 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 14:55:16.601209 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 14:55:16.601209 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 14:55:16.601209 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 26 14:55:16.601209 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 292/574] 10.222.0.46 () {56 vars in 1111 bytes} [Thu May 26 14:55:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:55:16.602865 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f287201-1c15-45a7-acb2-9b7694a0a7ac None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:16.640382 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f0454c-308b-4b08-b049-0efe61ed9a02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:16.641275 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f0454c-308b-4b08-b049-0efe61ed9a02 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:16.641507 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f0454c-308b-4b08-b049-0efe61ed9a02 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:16.641803 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f0454c-308b-4b08-b049-0efe61ed9a02 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:16.649680 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f0454c-308b-4b08-b049-0efe61ed9a02 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:16.651161 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f0454c-308b-4b08-b049-0efe61ed9a02 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:16.668871 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-71f0454c-308b-4b08-b049-0efe61ed9a02 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 14:55:16.668871 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:55:16.668871 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:55:16.668871 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:55:16.668871 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:55:16.668871 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:55:16.668871 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:55:16.668871 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:55:16.668871 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:55:16.668871 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:55:16.668871 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:55:16.668871 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:55:16.668871 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 14:55:16.668871 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 14:55:16.668871 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 14:55:16.669997 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 14:55:16.669997 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:55:16.669997 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:55:16.669997 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:55:16.669997 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:55:16.669997 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:55:16.669997 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:55:16.669997 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:55:16.669997 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 26 14:55:16.669997 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:55:16.669997 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 26 14:55:16.669997 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:55:16.669997 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:55:16.669997 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:55:16.669997 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 26 14:55:16.669997 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:55:16.669997 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 26 14:55:16.670929 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 14:55:16.670929 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 14:55:16.670929 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 14:55:16.670929 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 14:55:16.670929 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 14:55:16.670929 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 14:55:16.670929 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 14:55:16.670929 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 26 14:55:16.670929 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 283/575] 10.222.0.46 () {56 vars in 1111 bytes} [Thu May 26 14:55:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:55:16.671419 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd3753c-a6e9-476d-84ad-2257966e7c55 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:16.671698 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd3753c-a6e9-476d-84ad-2257966e7c55 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:16.672037 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd3753c-a6e9-476d-84ad-2257966e7c55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:16.684164 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:16.690576 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-edd3753c-a6e9-476d-84ad-2257966e7c55 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:55:16.692612 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-edd3753c-a6e9-476d-84ad-2257966e7c55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:55:16.754587 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f287201-1c15-45a7-acb2-9b7694a0a7ac None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:16.792080 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 284/576] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 14:55:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 14:55:16.792998 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd3753c-a6e9-476d-84ad-2257966e7c55 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:16.805481 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f118f557-8a63-4ca5-8024-871fbf679565 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:16.806682 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f118f557-8a63-4ca5-8024-871fbf679565 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:16.807105 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f118f557-8a63-4ca5-8024-871fbf679565 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:16.807512 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f118f557-8a63-4ca5-8024-871fbf679565 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:16.824194 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f118f557-8a63-4ca5-8024-871fbf679565 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:16.825689 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f118f557-8a63-4ca5-8024-871fbf679565 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:16.831853 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:16.846391 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-f118f557-8a63-4ca5-8024-871fbf679565 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 14:55:16.846391 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:55:16.846391 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:55:16.846391 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:55:16.846391 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:55:16.846391 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:55:16.846391 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:55:16.846391 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:55:16.846391 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:55:16.846391 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:55:16.846391 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:55:16.846391 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:55:16.846391 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 14:55:16.846391 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 14:55:16.846391 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 14:55:16.847754 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 14:55:16.847754 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:55:16.847754 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:55:16.847754 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:55:16.847754 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:55:16.847754 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:55:16.847754 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:55:16.847754 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:55:16.847754 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 26 14:55:16.847754 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:55:16.847754 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 26 14:55:16.847754 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:55:16.847754 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:55:16.847754 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:55:16.847754 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 26 14:55:16.847754 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:55:16.847754 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 26 14:55:16.849232 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 14:55:16.849232 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 14:55:16.849232 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 14:55:16.849232 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 26 14:55:16.849232 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 293/577] 10.222.0.46 () {56 vars in 1108 bytes} [Thu May 26 14:55:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:55:16.849232 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f118f557-8a63-4ca5-8024-871fbf679565 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:16.965809 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd3753c-a6e9-476d-84ad-2257966e7c55 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:16.976609 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f118f557-8a63-4ca5-8024-871fbf679565 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:16.979253 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:17.015842 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f373930-6418-46d0-bea5-2f29cae5132b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:17.016924 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f373930-6418-46d0-bea5-2f29cae5132b None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:17.017355 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f373930-6418-46d0-bea5-2f29cae5132b None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:17.017801 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f373930-6418-46d0-bea5-2f29cae5132b None admin] PATH_INFO: `/v3/users/nova` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:17.021558 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2bdae2-e49b-498b-b5a2-bf6cc7da8e8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:17.022243 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2bdae2-e49b-498b-b5a2-bf6cc7da8e8f None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:17.022533 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2bdae2-e49b-498b-b5a2-bf6cc7da8e8f None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:17.022817 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2bdae2-e49b-498b-b5a2-bf6cc7da8e8f None admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:17.023618 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2bdae2-e49b-498b-b5a2-bf6cc7da8e8f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:55:17.023933 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2bdae2-e49b-498b-b5a2-bf6cc7da8e8f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:17.025391 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2bdae2-e49b-498b-b5a2-bf6cc7da8e8f None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:17.027629 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f373930-6418-46d0-bea5-2f29cae5132b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:17.028802 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f373930-6418-46d0-bea5-2f29cae5132b None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:17.039119 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 294/578] 10.222.0.46 () {56 vars in 1148 bytes} [Thu May 26 14:55:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:55:17.039998 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2bdae2-e49b-498b-b5a2-bf6cc7da8e8f None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:17.044234 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-7f373930-6418-46d0-bea5-2f29cae5132b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 26 14:55:17.044234 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:55:17.044234 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:55:17.044234 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:55:17.044234 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:55:17.044234 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:55:17.044234 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:55:17.044234 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:55:17.044234 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:55:17.044234 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:55:17.044234 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:55:17.044234 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:55:17.044234 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 14:55:17.044234 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 14:55:17.044234 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 14:55:17.044234 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 14:55:17.045998 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:55:17.045998 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:55:17.045998 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 14:55:17.045998 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 14:55:17.045998 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 14:55:17.045998 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 14:55:17.045998 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:55:17.045998 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:55:17.045998 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:55:17.045998 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:55:17.045998 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:55:17.045998 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 26 14:55:17.045998 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:55:17.045998 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 26 14:55:17.045998 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:55:17.045998 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:55:17.045998 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:55:17.045998 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 26 14:55:17.047310 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:55:17.047310 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 26 14:55:17.047310 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 14:55:17.047310 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 14:55:17.047310 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 14:55:17.047310 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 14:55:17.047310 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 14:55:17.047310 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 14:55:17.047310 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 26 14:55:17.047310 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 26 14:55:17.047310 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 285/579] 10.222.0.46 () {56 vars in 1093 bytes} [Thu May 26 14:55:16 2022] GET /identity/v3/users/nova => generated 82 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:55:17.047310 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f373930-6418-46d0-bea5-2f29cae5132b None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:17.161807 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:17.175392 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f373930-6418-46d0-bea5-2f29cae5132b None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:17.190498 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2bdae2-e49b-498b-b5a2-bf6cc7da8e8f None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:17.211687 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3913df-9957-4f3a-a600-7d1c84d15a7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:17.212530 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3913df-9957-4f3a-a600-7d1c84d15a7a None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:17.218222 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3913df-9957-4f3a-a600-7d1c84d15a7a None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:17.218222 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3913df-9957-4f3a-a600-7d1c84d15a7a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:17.229305 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3913df-9957-4f3a-a600-7d1c84d15a7a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 14:55:17.233608 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bb1764-c27a-4539-bbbf-6a8f33701513 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:17.234194 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bb1764-c27a-4539-bbbf-6a8f33701513 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:17.234473 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bb1764-c27a-4539-bbbf-6a8f33701513 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:17.234738 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bb1764-c27a-4539-bbbf-6a8f33701513 None admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:17.235517 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bb1764-c27a-4539-bbbf-6a8f33701513 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:55:17.235824 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bb1764-c27a-4539-bbbf-6a8f33701513 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:17.237255 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bb1764-c27a-4539-bbbf-6a8f33701513 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:17.237834 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-df3913df-9957-4f3a-a600-7d1c84d15a7a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 26 14:55:17.237834 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:55:17.237834 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:55:17.237834 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:55:17.237834 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:55:17.237834 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:55:17.237834 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:55:17.237834 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:55:17.237834 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:55:17.237834 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:55:17.237834 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:55:17.237834 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:55:17.237834 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 14:55:17.237834 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 14:55:17.237834 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 14:55:17.237834 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 14:55:17.239068 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 14:55:17.239068 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 14:55:17.239068 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 14:55:17.239068 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 14:55:17.239068 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:55:17.239068 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:55:17.239068 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:55:17.239068 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:55:17.239068 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:55:17.239068 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:55:17.239068 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:55:17.239068 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 26 14:55:17.239068 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:55:17.239068 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 26 14:55:17.239068 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:55:17.239068 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:55:17.240279 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:55:17.240279 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 26 14:55:17.240279 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:55:17.240279 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 26 14:55:17.240279 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 14:55:17.240279 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 14:55:17.240279 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 14:55:17.240279 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 14:55:17.240279 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 14:55:17.240279 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 14:55:17.240279 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 26 14:55:17.240279 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 26 14:55:17.240279 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 286/580] 10.222.0.46 () {56 vars in 1102 bytes} [Thu May 26 14:55:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:55:17.240279 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3913df-9957-4f3a-a600-7d1c84d15a7a None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:17.255114 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 295/581] 10.222.0.46 () {56 vars in 1148 bytes} [Thu May 26 14:55:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:55:17.255792 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bb1764-c27a-4539-bbbf-6a8f33701513 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:17.292563 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:17.373759 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bb1764-c27a-4539-bbbf-6a8f33701513 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:17.380708 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3913df-9957-4f3a-a600-7d1c84d15a7a None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:17.423247 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04e3d86-a190-4abc-a3d7-ab7b2730c49b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:17.423827 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04e3d86-a190-4abc-a3d7-ab7b2730c49b None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:17.424149 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04e3d86-a190-4abc-a3d7-ab7b2730c49b None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:17.424464 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04e3d86-a190-4abc-a3d7-ab7b2730c49b None admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:17.425220 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f16ffb6-38e5-4deb-8c03-5c16fca6edce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:17.425387 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04e3d86-a190-4abc-a3d7-ab7b2730c49b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:55:17.425544 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04e3d86-a190-4abc-a3d7-ab7b2730c49b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:17.426810 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f16ffb6-38e5-4deb-8c03-5c16fca6edce None admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:17.427017 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04e3d86-a190-4abc-a3d7-ab7b2730c49b None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:17.427316 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f16ffb6-38e5-4deb-8c03-5c16fca6edce None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:17.427613 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f16ffb6-38e5-4deb-8c03-5c16fca6edce None admin] PATH_INFO: `/v3/projects/a17fc8227d4c4da6b2847b350e912ee3/users/0a436dbdee624f44ad76cca8416b0ad1/roles/eb76213820fa4c4e9b0a5d5ac2a08b41` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:17.442962 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 287/582] 10.222.0.46 () {56 vars in 1109 bytes} [Thu May 26 14:55:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:55:17.444102 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04e3d86-a190-4abc-a3d7-ab7b2730c49b None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:17.444299 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f16ffb6-38e5-4deb-8c03-5c16fca6edce None admin] RBAC: Authorizing `identity:create_grant(project_id=a17fc8227d4c4da6b2847b350e912ee3, user_id=0a436dbdee624f44ad76cca8416b0ad1, role_id=eb76213820fa4c4e9b0a5d5ac2a08b41)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:17.445692 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f16ffb6-38e5-4deb-8c03-5c16fca6edce None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:17.461880 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:17.468691 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 296/583] 10.222.0.46 () {58 vars in 1439 bytes} [Thu May 26 14:55:17 2022] PUT /identity/v3/projects/a17fc8227d4c4da6b2847b350e912ee3/users/0a436dbdee624f44ad76cca8416b0ad1/roles/eb76213820fa4c4e9b0a5d5ac2a08b41 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 14:55:17.469921 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f16ffb6-38e5-4deb-8c03-5c16fca6edce None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:17.601122 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:17.618358 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04e3d86-a190-4abc-a3d7-ab7b2730c49b None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:17.654227 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f16ffb6-38e5-4deb-8c03-5c16fca6edce None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:17.689226 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feba3ffe-141d-4596-82f3-0c313e7fb42d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:17.690752 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feba3ffe-141d-4596-82f3-0c313e7fb42d None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:17.691418 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feba3ffe-141d-4596-82f3-0c313e7fb42d None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:17.692142 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feba3ffe-141d-4596-82f3-0c313e7fb42d None admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:17.693632 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feba3ffe-141d-4596-82f3-0c313e7fb42d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:55:17.694241 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feba3ffe-141d-4596-82f3-0c313e7fb42d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:17.696620 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feba3ffe-141d-4596-82f3-0c313e7fb42d None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:17.711307 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11efb9c-8559-4920-8f3e-69d145ebb074 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:17.712089 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11efb9c-8559-4920-8f3e-69d145ebb074 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:17.712442 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11efb9c-8559-4920-8f3e-69d145ebb074 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:17.712791 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11efb9c-8559-4920-8f3e-69d145ebb074 None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:17.713575 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11efb9c-8559-4920-8f3e-69d145ebb074 None admin] RBAC: Adding query filter params (name=service) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:55:17.713917 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11efb9c-8559-4920-8f3e-69d145ebb074 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:17.715294 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11efb9c-8559-4920-8f3e-69d145ebb074 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:17.729815 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 297/584] 10.222.0.46 () {56 vars in 1103 bytes} [Thu May 26 14:55:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:55:17.729815 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11efb9c-8559-4920-8f3e-69d145ebb074 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:17.734415 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 288/585] 10.222.0.46 () {56 vars in 1133 bytes} [Thu May 26 14:55:17 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:55:17.735748 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feba3ffe-141d-4596-82f3-0c313e7fb42d None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:17.860141 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:17.889166 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11efb9c-8559-4920-8f3e-69d145ebb074 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:17.912394 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feba3ffe-141d-4596-82f3-0c313e7fb42d None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:17.945825 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561a4146-28ab-4198-b64c-61274701a21f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:17.946498 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561a4146-28ab-4198-b64c-61274701a21f None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:17.946728 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561a4146-28ab-4198-b64c-61274701a21f None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:17.947009 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561a4146-28ab-4198-b64c-61274701a21f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:17.947924 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561a4146-28ab-4198-b64c-61274701a21f None admin] RBAC: Adding query filter params (role.id=eb76213820fa4c4e9b0a5d5ac2a08b41, scope.project.id=a17fc8227d4c4da6b2847b350e912ee3, user.id=a4bea272624b45d8ac2ad27bd07df885) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:55:17.948809 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561a4146-28ab-4198-b64c-61274701a21f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:17.951186 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561a4146-28ab-4198-b64c-61274701a21f None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:17.956240 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:17.957691 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db24e29-3a0a-49fa-ada9-129e01d54477 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:17.958674 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db24e29-3a0a-49fa-ada9-129e01d54477 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:17.959004 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db24e29-3a0a-49fa-ada9-129e01d54477 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:17.959323 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db24e29-3a0a-49fa-ada9-129e01d54477 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:17.962242 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 298/586] 10.222.0.46 () {56 vars in 1374 bytes} [Thu May 26 14:55:17 2022] GET /identity/v3/role_assignments?user.id=a4bea272624b45d8ac2ad27bd07df885&scope.project.id=a17fc8227d4c4da6b2847b350e912ee3&role.id=eb76213820fa4c4e9b0a5d5ac2a08b41 => generated 259 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:55:17.964559 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561a4146-28ab-4198-b64c-61274701a21f None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:17.976581 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db24e29-3a0a-49fa-ada9-129e01d54477 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:17.978498 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db24e29-3a0a-49fa-ada9-129e01d54477 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:17.997101 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-5db24e29-3a0a-49fa-ada9-129e01d54477 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 26 14:55:17.997101 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:55:17.997101 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:55:17.997101 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:55:17.997101 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:55:17.997101 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:55:17.997101 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:55:17.997101 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:55:17.997101 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:55:17.997101 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:55:17.997101 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:55:17.997101 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:55:17.997101 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 14:55:17.997101 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 14:55:17.997101 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 14:55:17.997101 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 14:55:17.998308 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:55:17.998308 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:55:17.998308 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 14:55:17.998308 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 14:55:17.998308 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 14:55:17.998308 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 14:55:17.998308 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:55:17.998308 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:55:17.998308 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:55:17.998308 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:55:17.998308 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:55:17.998308 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 26 14:55:17.998308 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:55:17.998308 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 26 14:55:17.998308 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:55:17.998308 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:55:17.998308 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:55:17.998308 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 26 14:55:17.999449 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:55:17.999449 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 26 14:55:17.999449 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 14:55:17.999449 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 14:55:17.999449 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 14:55:17.999449 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 14:55:17.999449 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 14:55:17.999449 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 14:55:17.999449 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 26 14:55:17.999449 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 26 14:55:18.002027 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 289/587] 10.222.0.46 () {56 vars in 1099 bytes} [Thu May 26 14:55:17 2022] GET /identity/v3/users/cinder => generated 84 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:55:18.003172 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db24e29-3a0a-49fa-ada9-129e01d54477 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:18.142161 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561a4146-28ab-4198-b64c-61274701a21f None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:18.190657 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db24e29-3a0a-49fa-ada9-129e01d54477 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:18.191952 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0db966-c184-428b-ae30-e1ebd4d122a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:18.193250 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0db966-c184-428b-ae30-e1ebd4d122a4 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:18.193638 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0db966-c184-428b-ae30-e1ebd4d122a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:18.194143 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0db966-c184-428b-ae30-e1ebd4d122a4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:18.209248 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0db966-c184-428b-ae30-e1ebd4d122a4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:18.211360 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0db966-c184-428b-ae30-e1ebd4d122a4 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:18.231137 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:18.233561 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-ed0db966-c184-428b-ae30-e1ebd4d122a4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 14:55:18.233561 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:55:18.233561 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:55:18.233561 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:55:18.233561 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:55:18.233561 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:55:18.233561 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:55:18.233561 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:55:18.233561 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:55:18.233561 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:55:18.233561 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:55:18.233561 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:55:18.233561 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 14:55:18.233561 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 14:55:18.233561 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 14:55:18.234442 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 14:55:18.234442 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:55:18.234442 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:55:18.234442 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:55:18.234442 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:55:18.234442 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:55:18.234442 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:55:18.234442 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:55:18.234442 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 26 14:55:18.234442 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:55:18.234442 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 26 14:55:18.234442 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:55:18.234442 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:55:18.234442 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:55:18.234442 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 26 14:55:18.234442 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:55:18.234442 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 26 14:55:18.235334 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 14:55:18.235334 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 14:55:18.235334 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 14:55:18.235334 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 26 14:55:18.235334 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 299/588] 10.222.0.46 () {56 vars in 1108 bytes} [Thu May 26 14:55:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:55:18.236178 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0db966-c184-428b-ae30-e1ebd4d122a4 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:18.243901 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721beda1-0992-44a4-a09c-8335e89bae9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:18.244728 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721beda1-0992-44a4-a09c-8335e89bae9e None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:18.245649 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721beda1-0992-44a4-a09c-8335e89bae9e None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:18.246085 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721beda1-0992-44a4-a09c-8335e89bae9e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:18.259620 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721beda1-0992-44a4-a09c-8335e89bae9e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:18.261725 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721beda1-0992-44a4-a09c-8335e89bae9e None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:18.284720 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-721beda1-0992-44a4-a09c-8335e89bae9e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 14:55:18.284720 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:55:18.284720 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:55:18.284720 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:55:18.284720 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:55:18.284720 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:55:18.284720 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:55:18.284720 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:55:18.284720 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:55:18.284720 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:55:18.284720 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:55:18.284720 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:55:18.284720 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 14:55:18.284720 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 14:55:18.284720 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 14:55:18.287279 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 14:55:18.287279 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:55:18.287279 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:55:18.287279 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:55:18.287279 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:55:18.287279 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:55:18.287279 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:55:18.287279 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:55:18.287279 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 26 14:55:18.287279 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:55:18.287279 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 26 14:55:18.287279 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:55:18.287279 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:55:18.287279 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:55:18.287279 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 26 14:55:18.287279 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:55:18.287279 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 26 14:55:18.289985 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 14:55:18.289985 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 14:55:18.289985 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 14:55:18.289985 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 26 14:55:18.289985 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 290/589] 10.222.0.46 () {56 vars in 1108 bytes} [Thu May 26 14:55:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:55:18.289985 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721beda1-0992-44a4-a09c-8335e89bae9e None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:18.310370 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:18.367269 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0db966-c184-428b-ae30-e1ebd4d122a4 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:18.407477 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f523fec-b1ed-44c0-a0c8-2c430af39c2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:18.408483 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f523fec-b1ed-44c0-a0c8-2c430af39c2d None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:18.408483 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f523fec-b1ed-44c0-a0c8-2c430af39c2d None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:18.408794 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f523fec-b1ed-44c0-a0c8-2c430af39c2d None admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:18.409560 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f523fec-b1ed-44c0-a0c8-2c430af39c2d None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:55:18.409877 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f523fec-b1ed-44c0-a0c8-2c430af39c2d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:18.411460 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f523fec-b1ed-44c0-a0c8-2c430af39c2d None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:18.430050 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721beda1-0992-44a4-a09c-8335e89bae9e None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:18.436428 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 300/590] 10.222.0.46 () {56 vars in 1137 bytes} [Thu May 26 14:55:18 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:55:18.437732 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f523fec-b1ed-44c0-a0c8-2c430af39c2d None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:18.488557 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05702d4d-8179-4403-a979-964f55c0fe71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:18.489710 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05702d4d-8179-4403-a979-964f55c0fe71 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:18.490208 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05702d4d-8179-4403-a979-964f55c0fe71 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:18.490640 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05702d4d-8179-4403-a979-964f55c0fe71 None admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:18.491675 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05702d4d-8179-4403-a979-964f55c0fe71 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:55:18.492213 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05702d4d-8179-4403-a979-964f55c0fe71 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:18.493953 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05702d4d-8179-4403-a979-964f55c0fe71 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:18.508725 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 291/591] 10.222.0.46 () {56 vars in 1109 bytes} [Thu May 26 14:55:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:55:18.509980 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05702d4d-8179-4403-a979-964f55c0fe71 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:18.558705 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:18.580510 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f523fec-b1ed-44c0-a0c8-2c430af39c2d None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:18.593669 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:18.635974 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a1a5d7-3561-40b9-bb80-65f6c2d12583 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:18.637292 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a1a5d7-3561-40b9-bb80-65f6c2d12583 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:18.637940 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a1a5d7-3561-40b9-bb80-65f6c2d12583 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:18.638541 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a1a5d7-3561-40b9-bb80-65f6c2d12583 None admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:18.640018 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a1a5d7-3561-40b9-bb80-65f6c2d12583 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:55:18.640710 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a1a5d7-3561-40b9-bb80-65f6c2d12583 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:18.643362 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a1a5d7-3561-40b9-bb80-65f6c2d12583 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:18.652564 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05702d4d-8179-4403-a979-964f55c0fe71 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:18.669889 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 301/592] 10.222.0.46 () {56 vars in 1109 bytes} [Thu May 26 14:55:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:55:18.671637 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a1a5d7-3561-40b9-bb80-65f6c2d12583 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:18.706145 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53edf00-3875-419f-ac99-06da52fc71dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:18.707807 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53edf00-3875-419f-ac99-06da52fc71dc None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:18.708311 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53edf00-3875-419f-ac99-06da52fc71dc None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:18.708788 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53edf00-3875-419f-ac99-06da52fc71dc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:18.729105 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53edf00-3875-419f-ac99-06da52fc71dc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:18.732543 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d53edf00-3875-419f-ac99-06da52fc71dc None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:18.766971 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-d53edf00-3875-419f-ac99-06da52fc71dc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 14:55:18.766971 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:55:18.766971 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:55:18.766971 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:55:18.766971 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:55:18.766971 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:55:18.766971 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:55:18.766971 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:55:18.766971 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:55:18.766971 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:55:18.766971 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:55:18.766971 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:55:18.766971 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 14:55:18.766971 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 14:55:18.766971 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 14:55:18.767908 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 14:55:18.767908 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:55:18.767908 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:55:18.767908 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:55:18.767908 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:55:18.767908 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:55:18.767908 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:55:18.767908 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:55:18.767908 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 26 14:55:18.767908 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:55:18.767908 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 26 14:55:18.767908 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:55:18.767908 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:55:18.767908 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:55:18.767908 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 26 14:55:18.767908 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:55:18.767908 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 26 14:55:18.768790 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 14:55:18.768790 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 14:55:18.768790 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 14:55:18.768790 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 26 14:55:18.775478 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 292/593] 10.222.0.46 () {56 vars in 1108 bytes} [Thu May 26 14:55:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:55:18.778171 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53edf00-3875-419f-ac99-06da52fc71dc None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:18.810545 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a1a5d7-3561-40b9-bb80-65f6c2d12583 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:18.849405 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2da330e-9be0-4028-a181-476b49d3050d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:18.850709 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2da330e-9be0-4028-a181-476b49d3050d None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:18.851228 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2da330e-9be0-4028-a181-476b49d3050d None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:18.851705 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2da330e-9be0-4028-a181-476b49d3050d None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:18.860855 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2da330e-9be0-4028-a181-476b49d3050d None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:18.862768 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2da330e-9be0-4028-a181-476b49d3050d None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:18.879006 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-b2da330e-9be0-4028-a181-476b49d3050d None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 26 14:55:18.879006 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:55:18.879006 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:55:18.879006 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:55:18.879006 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:55:18.879006 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:55:18.879006 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:55:18.879006 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:55:18.879006 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:55:18.879006 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:55:18.879006 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:55:18.879006 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:55:18.879006 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 14:55:18.879006 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 14:55:18.879006 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 14:55:18.879006 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 14:55:18.880257 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:55:18.880257 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:55:18.880257 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 14:55:18.880257 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 14:55:18.880257 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 14:55:18.880257 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 14:55:18.880257 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:55:18.880257 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:55:18.880257 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:55:18.880257 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:55:18.880257 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:55:18.880257 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 26 14:55:18.880257 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:55:18.880257 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 26 14:55:18.880257 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:55:18.880257 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:55:18.880257 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:55:18.880257 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 26 14:55:18.881196 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:55:18.881196 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 26 14:55:18.881196 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 14:55:18.881196 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 14:55:18.881196 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 14:55:18.881196 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 14:55:18.881196 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 14:55:18.881196 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 14:55:18.881196 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 26 14:55:18.881196 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 26 14:55:18.884057 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 302/594] 10.222.0.46 () {56 vars in 1102 bytes} [Thu May 26 14:55:18 2022] GET /identity/v3/users/neutron => generated 85 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:55:18.885147 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2da330e-9be0-4028-a181-476b49d3050d None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:18.920841 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:18.962064 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:18.980539 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53edf00-3875-419f-ac99-06da52fc71dc None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:19.005388 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2da330e-9be0-4028-a181-476b49d3050d None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:19.042992 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3019473c-a76c-49c6-9218-2f919b12f7b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:19.043913 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3019473c-a76c-49c6-9218-2f919b12f7b2 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:19.044375 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3019473c-a76c-49c6-9218-2f919b12f7b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:19.044798 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3019473c-a76c-49c6-9218-2f919b12f7b2 None admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:19.045763 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3019473c-a76c-49c6-9218-2f919b12f7b2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:55:19.046233 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3019473c-a76c-49c6-9218-2f919b12f7b2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:19.047766 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3019473c-a76c-49c6-9218-2f919b12f7b2 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:19.048474 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e814b2b4-8126-4533-8824-88cd4d9fac54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:19.050150 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e814b2b4-8126-4533-8824-88cd4d9fac54 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:19.050784 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e814b2b4-8126-4533-8824-88cd4d9fac54 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:19.051536 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e814b2b4-8126-4533-8824-88cd4d9fac54 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:19.062911 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 303/595] 10.222.0.46 () {56 vars in 1109 bytes} [Thu May 26 14:55:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:55:19.064050 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3019473c-a76c-49c6-9218-2f919b12f7b2 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:19.075105 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e814b2b4-8126-4533-8824-88cd4d9fac54 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:19.078902 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e814b2b4-8126-4533-8824-88cd4d9fac54 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:19.113561 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-e814b2b4-8126-4533-8824-88cd4d9fac54 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 14:55:19.113561 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:55:19.113561 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:55:19.113561 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:55:19.113561 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:55:19.113561 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:55:19.113561 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:55:19.113561 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:55:19.113561 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:55:19.113561 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:55:19.113561 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:55:19.113561 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:55:19.113561 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 14:55:19.113561 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 14:55:19.113561 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 14:55:19.115559 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 14:55:19.115559 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:55:19.115559 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:55:19.115559 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:55:19.115559 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:55:19.115559 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:55:19.115559 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:55:19.115559 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:55:19.115559 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 26 14:55:19.115559 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:55:19.115559 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 26 14:55:19.115559 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:55:19.115559 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:55:19.115559 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:55:19.115559 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 26 14:55:19.115559 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:55:19.115559 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 26 14:55:19.117300 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 14:55:19.117300 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 14:55:19.117300 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 14:55:19.117300 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 14:55:19.117300 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 14:55:19.117300 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 14:55:19.117300 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 14:55:19.117300 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 26 14:55:19.126214 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 293/596] 10.222.0.46 () {56 vars in 1111 bytes} [Thu May 26 14:55:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 343 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:55:19.126214 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e814b2b4-8126-4533-8824-88cd4d9fac54 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:19.222159 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:19.235407 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3019473c-a76c-49c6-9218-2f919b12f7b2 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:19.281400 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fffdf78-35d0-4155-87d3-2a95a7984023 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:19.282488 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fffdf78-35d0-4155-87d3-2a95a7984023 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:19.282998 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fffdf78-35d0-4155-87d3-2a95a7984023 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:19.283474 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fffdf78-35d0-4155-87d3-2a95a7984023 None admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:19.284570 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fffdf78-35d0-4155-87d3-2a95a7984023 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:55:19.285092 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fffdf78-35d0-4155-87d3-2a95a7984023 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:19.286856 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fffdf78-35d0-4155-87d3-2a95a7984023 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:19.304652 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e814b2b4-8126-4533-8824-88cd4d9fac54 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:19.308502 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 304/597] 10.222.0.46 () {56 vars in 1139 bytes} [Thu May 26 14:55:19 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:55:19.309749 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fffdf78-35d0-4155-87d3-2a95a7984023 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:19.361070 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ce52ac-6fe9-40d6-a39b-668ed3c4a69d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:19.361923 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ce52ac-6fe9-40d6-a39b-668ed3c4a69d None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:19.362203 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ce52ac-6fe9-40d6-a39b-668ed3c4a69d None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:19.362590 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ce52ac-6fe9-40d6-a39b-668ed3c4a69d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:19.373337 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ce52ac-6fe9-40d6-a39b-668ed3c4a69d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:19.374876 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ce52ac-6fe9-40d6-a39b-668ed3c4a69d None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:19.398769 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-59ce52ac-6fe9-40d6-a39b-668ed3c4a69d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 14:55:19.398769 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:55:19.398769 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:55:19.398769 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:55:19.398769 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:55:19.398769 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:55:19.398769 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:55:19.398769 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:55:19.398769 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:55:19.398769 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:55:19.398769 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:55:19.398769 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:55:19.398769 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 14:55:19.398769 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 14:55:19.398769 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 14:55:19.399807 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 14:55:19.399807 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:55:19.399807 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:55:19.399807 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:55:19.399807 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:55:19.399807 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:55:19.399807 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:55:19.399807 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:55:19.399807 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 26 14:55:19.399807 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:55:19.399807 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 26 14:55:19.399807 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:55:19.399807 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:55:19.399807 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:55:19.399807 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 26 14:55:19.399807 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:55:19.399807 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 26 14:55:19.400764 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 14:55:19.400764 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 14:55:19.400764 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 14:55:19.400764 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 14:55:19.400764 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 14:55:19.400764 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 14:55:19.400764 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 14:55:19.400764 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 26 14:55:19.407351 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 294/598] 10.222.0.46 () {56 vars in 1111 bytes} [Thu May 26 14:55:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:55:19.408807 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ce52ac-6fe9-40d6-a39b-668ed3c4a69d None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:19.429104 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fffdf78-35d0-4155-87d3-2a95a7984023 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:19.467663 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472a87b9-e67e-4fb4-8751-dc9611deec9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:19.468782 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472a87b9-e67e-4fb4-8751-dc9611deec9c None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:19.469271 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472a87b9-e67e-4fb4-8751-dc9611deec9c None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:19.469948 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:19.470061 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472a87b9-e67e-4fb4-8751-dc9611deec9c None admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:19.471234 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472a87b9-e67e-4fb4-8751-dc9611deec9c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:55:19.471756 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472a87b9-e67e-4fb4-8751-dc9611deec9c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:19.473467 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472a87b9-e67e-4fb4-8751-dc9611deec9c None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:19.491695 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 305/599] 10.222.0.46 () {56 vars in 1148 bytes} [Thu May 26 14:55:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:55:19.493023 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472a87b9-e67e-4fb4-8751-dc9611deec9c None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:19.506523 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:19.580828 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ce52ac-6fe9-40d6-a39b-668ed3c4a69d None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:19.644162 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661422c0-51c2-41f7-b1c0-9fda092f3f16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:19.645072 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661422c0-51c2-41f7-b1c0-9fda092f3f16 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:19.645547 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661422c0-51c2-41f7-b1c0-9fda092f3f16 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:19.645774 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661422c0-51c2-41f7-b1c0-9fda092f3f16 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:19.657134 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472a87b9-e67e-4fb4-8751-dc9611deec9c None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:19.664853 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661422c0-51c2-41f7-b1c0-9fda092f3f16 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:19.667416 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661422c0-51c2-41f7-b1c0-9fda092f3f16 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:19.694565 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-661422c0-51c2-41f7-b1c0-9fda092f3f16 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 14:55:19.694565 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:55:19.694565 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:55:19.694565 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:55:19.694565 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:55:19.694565 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:55:19.694565 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:55:19.694565 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:55:19.694565 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:55:19.694565 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:55:19.694565 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:55:19.694565 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:55:19.694565 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 14:55:19.694565 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 14:55:19.694565 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 14:55:19.695557 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 14:55:19.695557 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:55:19.695557 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:55:19.695557 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:55:19.695557 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:55:19.695557 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:55:19.695557 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:55:19.695557 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:55:19.695557 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 26 14:55:19.695557 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:55:19.695557 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 26 14:55:19.695557 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:55:19.695557 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:55:19.695557 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:55:19.695557 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 26 14:55:19.695557 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:55:19.695557 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 26 14:55:19.696446 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 14:55:19.696446 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 14:55:19.696446 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 14:55:19.696446 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 26 14:55:19.697133 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 295/600] 10.222.0.46 () {56 vars in 1108 bytes} [Thu May 26 14:55:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:55:19.698941 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661422c0-51c2-41f7-b1c0-9fda092f3f16 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:19.699399 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdf25dc-1d10-4e39-b1ac-649c9cc35f97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:19.700319 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdf25dc-1d10-4e39-b1ac-649c9cc35f97 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:19.700801 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdf25dc-1d10-4e39-b1ac-649c9cc35f97 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:19.701272 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdf25dc-1d10-4e39-b1ac-649c9cc35f97 None admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:19.702322 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdf25dc-1d10-4e39-b1ac-649c9cc35f97 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:55:19.702896 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdf25dc-1d10-4e39-b1ac-649c9cc35f97 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:19.704734 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdf25dc-1d10-4e39-b1ac-649c9cc35f97 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:19.719866 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 306/601] 10.222.0.46 () {56 vars in 1148 bytes} [Thu May 26 14:55:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:55:19.721074 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdf25dc-1d10-4e39-b1ac-649c9cc35f97 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:19.810625 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:19.811946 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:19.864987 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661422c0-51c2-41f7-b1c0-9fda092f3f16 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:19.884607 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdf25dc-1d10-4e39-b1ac-649c9cc35f97 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:19.924757 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b427ef77-0eb7-4f72-a653-34acdce489b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:19.926255 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b427ef77-0eb7-4f72-a653-34acdce489b1 None admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:19.926788 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b427ef77-0eb7-4f72-a653-34acdce489b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:19.927598 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b427ef77-0eb7-4f72-a653-34acdce489b1 None admin] PATH_INFO: `/v3/projects/a17fc8227d4c4da6b2847b350e912ee3/users/d6639353c70449d1b3d10cd6487c3b00/roles/eb76213820fa4c4e9b0a5d5ac2a08b41` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:19.929094 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa111f86-eb5d-43ea-a941-8d3aa9ac5baa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:19.929783 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa111f86-eb5d-43ea-a941-8d3aa9ac5baa None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:19.930069 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa111f86-eb5d-43ea-a941-8d3aa9ac5baa None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:19.930347 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa111f86-eb5d-43ea-a941-8d3aa9ac5baa None admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:19.931084 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa111f86-eb5d-43ea-a941-8d3aa9ac5baa None admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:55:19.931393 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa111f86-eb5d-43ea-a941-8d3aa9ac5baa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:19.933337 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa111f86-eb5d-43ea-a941-8d3aa9ac5baa None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:19.949720 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 307/602] 10.222.0.46 () {56 vars in 1109 bytes} [Thu May 26 14:55:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:55:19.950039 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b427ef77-0eb7-4f72-a653-34acdce489b1 None admin] RBAC: Authorizing `identity:create_grant(project_id=a17fc8227d4c4da6b2847b350e912ee3, user_id=d6639353c70449d1b3d10cd6487c3b00, role_id=eb76213820fa4c4e9b0a5d5ac2a08b41)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:19.950628 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa111f86-eb5d-43ea-a941-8d3aa9ac5baa None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:19.952189 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b427ef77-0eb7-4f72-a653-34acdce489b1 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:19.972752 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 296/603] 10.222.0.46 () {58 vars in 1439 bytes} [Thu May 26 14:55:19 2022] PUT /identity/v3/projects/a17fc8227d4c4da6b2847b350e912ee3/users/d6639353c70449d1b3d10cd6487c3b00/roles/eb76213820fa4c4e9b0a5d5ac2a08b41 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 14:55:19.974038 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b427ef77-0eb7-4f72-a653-34acdce489b1 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:20.148523 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b427ef77-0eb7-4f72-a653-34acdce489b1 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:20.175604 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:20.212697 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74039791-fdb4-48ba-bfdf-3dae0b2055a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:20.214009 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74039791-fdb4-48ba-bfdf-3dae0b2055a4 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:20.215226 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74039791-fdb4-48ba-bfdf-3dae0b2055a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:20.215866 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74039791-fdb4-48ba-bfdf-3dae0b2055a4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:20.223998 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa111f86-eb5d-43ea-a941-8d3aa9ac5baa None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:20.225100 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74039791-fdb4-48ba-bfdf-3dae0b2055a4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:20.227299 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74039791-fdb4-48ba-bfdf-3dae0b2055a4 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:20.247665 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-74039791-fdb4-48ba-bfdf-3dae0b2055a4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 14:55:20.247665 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:55:20.247665 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:55:20.247665 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:55:20.247665 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:55:20.247665 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:55:20.247665 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:55:20.247665 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:55:20.247665 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:55:20.247665 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:55:20.247665 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:55:20.247665 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:55:20.247665 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 14:55:20.247665 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 14:55:20.247665 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 14:55:20.248662 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 14:55:20.248662 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:55:20.248662 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:55:20.248662 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:55:20.248662 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:55:20.248662 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:55:20.248662 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:55:20.248662 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:55:20.248662 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 26 14:55:20.248662 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:55:20.248662 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 26 14:55:20.248662 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:55:20.248662 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:55:20.248662 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:55:20.248662 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 26 14:55:20.248662 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:55:20.248662 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 26 14:55:20.249588 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 14:55:20.249588 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 14:55:20.249588 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 14:55:20.249588 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 14:55:20.249588 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 14:55:20.249588 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 14:55:20.249588 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 14:55:20.249588 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 26 14:55:20.253101 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 297/604] 10.222.0.46 () {56 vars in 1111 bytes} [Thu May 26 14:55:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:55:20.259702 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74039791-fdb4-48ba-bfdf-3dae0b2055a4 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:20.299984 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:20.304544 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f99e962-5c05-4996-9ab7-ef7f48be3874 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:20.305674 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f99e962-5c05-4996-9ab7-ef7f48be3874 None admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:20.305957 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f99e962-5c05-4996-9ab7-ef7f48be3874 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:20.306242 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f99e962-5c05-4996-9ab7-ef7f48be3874 None admin] PATH_INFO: `/v3/projects/a17fc8227d4c4da6b2847b350e912ee3/users/d5b5b35614a547d783e6e77ddb8c1c03/roles/eb76213820fa4c4e9b0a5d5ac2a08b41` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:20.331430 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f99e962-5c05-4996-9ab7-ef7f48be3874 None admin] RBAC: Authorizing `identity:create_grant(project_id=a17fc8227d4c4da6b2847b350e912ee3, user_id=d5b5b35614a547d783e6e77ddb8c1c03, role_id=eb76213820fa4c4e9b0a5d5ac2a08b41)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:20.334990 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f99e962-5c05-4996-9ab7-ef7f48be3874 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:20.357707 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 308/605] 10.222.0.46 () {58 vars in 1439 bytes} [Thu May 26 14:55:19 2022] PUT /identity/v3/projects/a17fc8227d4c4da6b2847b350e912ee3/users/d5b5b35614a547d783e6e77ddb8c1c03/roles/eb76213820fa4c4e9b0a5d5ac2a08b41 => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 14:55:20.431219 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74039791-fdb4-48ba-bfdf-3dae0b2055a4 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:20.438770 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b915b07-adaa-4a7b-8d0c-34826f64052d None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:20.439544 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b915b07-adaa-4a7b-8d0c-34826f64052d None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:20.439861 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b915b07-adaa-4a7b-8d0c-34826f64052d None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:20.441753 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 309/606] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 14:55:20 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 14:55:20.449554 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6341779f-8252-4bd1-b436-7a3a60e3c3b1 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:20.450197 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6341779f-8252-4bd1-b436-7a3a60e3c3b1 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:20.450766 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6341779f-8252-4bd1-b436-7a3a60e3c3b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:20.469435 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-6341779f-8252-4bd1-b436-7a3a60e3c3b1 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:55:20.471161 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-6341779f-8252-4bd1-b436-7a3a60e3c3b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:55:20.487981 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d8efde4-ece8-44c3-b84d-db0fa5d10889 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:20.489526 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8efde4-ece8-44c3-b84d-db0fa5d10889 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:20.489820 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8efde4-ece8-44c3-b84d-db0fa5d10889 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:20.490100 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d8efde4-ece8-44c3-b84d-db0fa5d10889 None admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:20.491222 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8efde4-ece8-44c3-b84d-db0fa5d10889 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:55:20.491540 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8efde4-ece8-44c3-b84d-db0fa5d10889 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:20.493349 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d8efde4-ece8-44c3-b84d-db0fa5d10889 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:20.514208 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 298/607] 10.222.0.46 () {56 vars in 1148 bytes} [Thu May 26 14:55:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 14:55:20.517430 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc9033b-001e-409e-9447-eb5399be5363 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:20.518163 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc9033b-001e-409e-9447-eb5399be5363 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:20.518992 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc9033b-001e-409e-9447-eb5399be5363 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:20.523955 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 299/608] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 14:55:20 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 14:55:20.526048 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc9033b-001e-409e-9447-eb5399be5363 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:20.572733 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:20.591840 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 310/609] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 14:55:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 14:55:20.593658 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7e22b4-a1a4-4f99-8e2a-d914a84c48cd None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:20.594177 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7e22b4-a1a4-4f99-8e2a-d914a84c48cd None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:20.594658 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f7e22b4-a1a4-4f99-8e2a-d914a84c48cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:20.633644 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-7f7e22b4-a1a4-4f99-8e2a-d914a84c48cd None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:55:20.635705 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-7f7e22b4-a1a4-4f99-8e2a-d914a84c48cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:55:20.685193 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc9033b-001e-409e-9447-eb5399be5363 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:20.738232 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2effab83-02bf-408b-880e-fa115bd5b37e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:20.739354 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2effab83-02bf-408b-880e-fa115bd5b37e None admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:20.739762 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2effab83-02bf-408b-880e-fa115bd5b37e None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:20.740165 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2effab83-02bf-408b-880e-fa115bd5b37e None admin] PATH_INFO: `/v3/projects/a17fc8227d4c4da6b2847b350e912ee3/users/0156a2f6f3794f479e76b85bdbbe5c26/roles/eb76213820fa4c4e9b0a5d5ac2a08b41` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:20.761998 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 311/610] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 14:55:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 14:55:20.763440 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7e22b4-a1a4-4f99-8e2a-d914a84c48cd None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:20.775776 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2effab83-02bf-408b-880e-fa115bd5b37e None admin] RBAC: Authorizing `identity:create_grant(project_id=a17fc8227d4c4da6b2847b350e912ee3, user_id=0156a2f6f3794f479e76b85bdbbe5c26, role_id=eb76213820fa4c4e9b0a5d5ac2a08b41)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:20.779694 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2effab83-02bf-408b-880e-fa115bd5b37e None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:20.805930 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 300/611] 10.222.0.46 () {58 vars in 1439 bytes} [Thu May 26 14:55:20 2022] PUT /identity/v3/projects/a17fc8227d4c4da6b2847b350e912ee3/users/0156a2f6f3794f479e76b85bdbbe5c26/roles/eb76213820fa4c4e9b0a5d5ac2a08b41 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 14:55:20.806855 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2effab83-02bf-408b-880e-fa115bd5b37e None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:20.889977 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:20.933564 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:20.952475 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2effab83-02bf-408b-880e-fa115bd5b37e None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:21.001970 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3adb7169-610a-4a1f-a440-fb99163ae806 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:21.003184 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adb7169-610a-4a1f-a440-fb99163ae806 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:21.003700 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adb7169-610a-4a1f-a440-fb99163ae806 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:21.003970 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adb7169-610a-4a1f-a440-fb99163ae806 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:21.030549 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3adb7169-610a-4a1f-a440-fb99163ae806 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 14:55:21.041487 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-3adb7169-610a-4a1f-a440-fb99163ae806 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 26 14:55:21.041487 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:55:21.041487 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:55:21.041487 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:55:21.041487 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:55:21.041487 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:55:21.041487 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:55:21.041487 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:55:21.041487 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:55:21.041487 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:55:21.041487 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:55:21.041487 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:55:21.041487 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 14:55:21.041487 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 14:55:21.041487 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 14:55:21.041487 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 14:55:21.043045 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 14:55:21.043045 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 14:55:21.043045 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 14:55:21.043045 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 14:55:21.043045 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:55:21.043045 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:55:21.043045 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:55:21.043045 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:55:21.043045 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:55:21.043045 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:55:21.043045 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:55:21.043045 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 26 14:55:21.043045 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:55:21.043045 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 26 14:55:21.043045 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:55:21.043045 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:55:21.043879 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:55:21.043879 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 26 14:55:21.043879 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:55:21.043879 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 26 14:55:21.043879 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 14:55:21.043879 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 14:55:21.043879 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 14:55:21.043879 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 14:55:21.043879 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 14:55:21.043879 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 14:55:21.043879 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 26 14:55:21.043879 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 26 14:55:21.055437 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 301/612] 10.222.0.46 () {56 vars in 1102 bytes} [Thu May 26 14:55:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:55:21.059044 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f7e22b4-a1a4-4f99-8e2a-d914a84c48cd None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:21.062200 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3adb7169-610a-4a1f-a440-fb99163ae806 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:21.141526 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b6fb5a-a473-4912-b719-9d325f92ba75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:21.142516 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b6fb5a-a473-4912-b719-9d325f92ba75 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:21.143525 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b6fb5a-a473-4912-b719-9d325f92ba75 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:21.144145 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b6fb5a-a473-4912-b719-9d325f92ba75 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:21.158220 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b6fb5a-a473-4912-b719-9d325f92ba75 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 14:55:21.168765 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-b0b6fb5a-a473-4912-b719-9d325f92ba75 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 26 14:55:21.168765 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:55:21.168765 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:55:21.168765 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:55:21.168765 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:55:21.168765 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:55:21.168765 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:55:21.168765 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:55:21.168765 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:55:21.168765 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:55:21.168765 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:55:21.168765 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:55:21.168765 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 14:55:21.168765 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 14:55:21.168765 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 14:55:21.168765 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 14:55:21.170664 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 14:55:21.170664 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 14:55:21.170664 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 14:55:21.170664 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 14:55:21.170664 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:55:21.170664 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:55:21.170664 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:55:21.170664 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:55:21.170664 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:55:21.170664 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:55:21.170664 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:55:21.170664 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 26 14:55:21.170664 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:55:21.170664 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 26 14:55:21.170664 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:55:21.170664 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:55:21.171704 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:55:21.171704 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 26 14:55:21.171704 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:55:21.171704 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 26 14:55:21.171704 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 14:55:21.171704 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 14:55:21.171704 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 14:55:21.171704 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 14:55:21.171704 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 14:55:21.171704 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 14:55:21.171704 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 26 14:55:21.171704 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 26 14:55:21.173389 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 312/613] 10.222.0.46 () {56 vars in 1102 bytes} [Thu May 26 14:55:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 411 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:55:21.179225 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b6fb5a-a473-4912-b719-9d325f92ba75 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:21.255532 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3adb7169-610a-4a1f-a440-fb99163ae806 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:21.359110 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c067b3-d83b-421d-860f-4be648371242 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:21.360033 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c067b3-d83b-421d-860f-4be648371242 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:21.360431 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c067b3-d83b-421d-860f-4be648371242 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:21.360824 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c067b3-d83b-421d-860f-4be648371242 None admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:21.361986 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c067b3-d83b-421d-860f-4be648371242 None admin] RBAC: Adding query filter params (name=service) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:55:21.362423 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c067b3-d83b-421d-860f-4be648371242 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:21.364418 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c067b3-d83b-421d-860f-4be648371242 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:21.381214 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 302/614] 10.222.0.46 () {56 vars in 1103 bytes} [Thu May 26 14:55:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 14:55:21.388889 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c067b3-d83b-421d-860f-4be648371242 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:21.394694 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:21.399002 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b6fb5a-a473-4912-b719-9d325f92ba75 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:21.413482 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:21.438863 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a43f7a-eb54-4ddd-94fc-347bea54b0d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:21.439651 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a43f7a-eb54-4ddd-94fc-347bea54b0d6 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:21.439882 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a43f7a-eb54-4ddd-94fc-347bea54b0d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:21.440190 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a43f7a-eb54-4ddd-94fc-347bea54b0d6 None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:21.440999 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a43f7a-eb54-4ddd-94fc-347bea54b0d6 None admin] RBAC: Adding query filter params (name=service) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:55:21.441338 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a43f7a-eb54-4ddd-94fc-347bea54b0d6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:21.442768 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a43f7a-eb54-4ddd-94fc-347bea54b0d6 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:21.457487 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 313/615] 10.222.0.46 () {56 vars in 1103 bytes} [Thu May 26 14:55:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:55:21.457909 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a43f7a-eb54-4ddd-94fc-347bea54b0d6 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:21.569850 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a43f7a-eb54-4ddd-94fc-347bea54b0d6 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:21.607276 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc0277c-78ac-4f13-bd4d-68f30e1cb760 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:21.608128 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc0277c-78ac-4f13-bd4d-68f30e1cb760 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:21.608411 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc0277c-78ac-4f13-bd4d-68f30e1cb760 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:21.608683 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc0277c-78ac-4f13-bd4d-68f30e1cb760 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:21.620511 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc0277c-78ac-4f13-bd4d-68f30e1cb760 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:21.622071 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc0277c-78ac-4f13-bd4d-68f30e1cb760 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:21.641712 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-2fc0277c-78ac-4f13-bd4d-68f30e1cb760 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 14:55:21.641712 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:55:21.641712 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:55:21.641712 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:55:21.641712 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:55:21.641712 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:55:21.641712 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:55:21.641712 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:55:21.641712 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:55:21.641712 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:55:21.641712 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:55:21.641712 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:55:21.641712 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 14:55:21.641712 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 14:55:21.641712 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 14:55:21.642846 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 14:55:21.642846 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:55:21.642846 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:55:21.642846 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:55:21.642846 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:55:21.642846 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:55:21.642846 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:55:21.642846 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:55:21.642846 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 26 14:55:21.642846 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:55:21.642846 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 26 14:55:21.642846 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:55:21.642846 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:55:21.642846 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:55:21.642846 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 26 14:55:21.642846 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:55:21.642846 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 26 14:55:21.643732 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 14:55:21.643732 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 14:55:21.643732 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 14:55:21.643732 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 26 14:55:21.643732 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 314/616] 10.222.0.46 () {56 vars in 1108 bytes} [Thu May 26 14:55:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:55:21.648710 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc0277c-78ac-4f13-bd4d-68f30e1cb760 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:21.680740 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:21.730104 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c067b3-d83b-421d-860f-4be648371242 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:21.814102 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aca469a-874f-458b-bf7b-c5202b4baf23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:21.815194 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aca469a-874f-458b-bf7b-c5202b4baf23 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:21.815627 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aca469a-874f-458b-bf7b-c5202b4baf23 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:21.816069 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aca469a-874f-458b-bf7b-c5202b4baf23 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:21.819350 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc0277c-78ac-4f13-bd4d-68f30e1cb760 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:21.845203 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aca469a-874f-458b-bf7b-c5202b4baf23 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:21.847085 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aca469a-874f-458b-bf7b-c5202b4baf23 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:21.859583 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a29a4c2-56ae-4e56-bc1c-e1060761143d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:21.860236 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a29a4c2-56ae-4e56-bc1c-e1060761143d None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:21.860518 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a29a4c2-56ae-4e56-bc1c-e1060761143d None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:21.860788 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a29a4c2-56ae-4e56-bc1c-e1060761143d None admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:21.861540 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a29a4c2-56ae-4e56-bc1c-e1060761143d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:55:21.861848 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a29a4c2-56ae-4e56-bc1c-e1060761143d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:21.863226 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a29a4c2-56ae-4e56-bc1c-e1060761143d None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:21.880574 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 315/617] 10.222.0.46 () {56 vars in 1109 bytes} [Thu May 26 14:55:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 14:55:21.882612 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a29a4c2-56ae-4e56-bc1c-e1060761143d None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:21.890061 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-6aca469a-874f-458b-bf7b-c5202b4baf23 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 14:55:21.890061 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:55:21.890061 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:55:21.890061 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:55:21.890061 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:55:21.890061 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:55:21.890061 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:55:21.890061 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:55:21.890061 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:55:21.890061 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:55:21.890061 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:55:21.890061 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:55:21.890061 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 14:55:21.890061 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 14:55:21.890061 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 14:55:21.891031 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 14:55:21.891031 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:55:21.891031 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:55:21.891031 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:55:21.891031 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:55:21.891031 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:55:21.891031 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:55:21.891031 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:55:21.891031 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 26 14:55:21.891031 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:55:21.891031 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 26 14:55:21.891031 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:55:21.891031 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:55:21.891031 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:55:21.891031 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 26 14:55:21.891031 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:55:21.891031 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 26 14:55:21.892123 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 14:55:21.892123 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 14:55:21.892123 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 14:55:21.892123 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 26 14:55:21.895450 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 303/618] 10.222.0.46 () {56 vars in 1108 bytes} [Thu May 26 14:55:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 507 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:55:21.904317 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aca469a-874f-458b-bf7b-c5202b4baf23 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:22.008140 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:22.021873 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:22.065115 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a29a4c2-56ae-4e56-bc1c-e1060761143d None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:22.136812 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aca469a-874f-458b-bf7b-c5202b4baf23 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:22.158207 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8044a372-e604-4c68-94aa-2e22eb6f8a8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:22.159043 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8044a372-e604-4c68-94aa-2e22eb6f8a8c None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:22.159328 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8044a372-e604-4c68-94aa-2e22eb6f8a8c None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:22.159612 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8044a372-e604-4c68-94aa-2e22eb6f8a8c None admin] PATH_INFO: `/v3/users/glance` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:22.175002 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6710e0b9-2daa-41d7-8b58-d5cd65685cdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:22.176061 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6710e0b9-2daa-41d7-8b58-d5cd65685cdd None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:22.176562 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6710e0b9-2daa-41d7-8b58-d5cd65685cdd None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:22.177038 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6710e0b9-2daa-41d7-8b58-d5cd65685cdd None admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:22.177675 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8044a372-e604-4c68-94aa-2e22eb6f8a8c None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:22.178258 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6710e0b9-2daa-41d7-8b58-d5cd65685cdd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:55:22.178785 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6710e0b9-2daa-41d7-8b58-d5cd65685cdd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:22.179170 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8044a372-e604-4c68-94aa-2e22eb6f8a8c None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:22.180820 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6710e0b9-2daa-41d7-8b58-d5cd65685cdd None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:22.197984 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 304/619] 10.222.0.46 () {56 vars in 1109 bytes} [Thu May 26 14:55:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:55:22.204110 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6710e0b9-2daa-41d7-8b58-d5cd65685cdd None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:22.218552 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-8044a372-e604-4c68-94aa-2e22eb6f8a8c None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 26 14:55:22.218552 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:55:22.218552 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:55:22.218552 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:55:22.218552 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:55:22.218552 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:55:22.218552 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:55:22.218552 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:55:22.218552 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:55:22.218552 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:55:22.218552 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:55:22.218552 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:55:22.218552 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 14:55:22.218552 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 14:55:22.218552 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 14:55:22.218552 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 14:55:22.219568 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:55:22.219568 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:55:22.219568 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 14:55:22.219568 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 14:55:22.219568 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 14:55:22.219568 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 14:55:22.219568 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:55:22.219568 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:55:22.219568 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:55:22.219568 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:55:22.219568 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:55:22.219568 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 26 14:55:22.219568 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:55:22.219568 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 26 14:55:22.219568 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:55:22.219568 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:55:22.219568 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:55:22.219568 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 26 14:55:22.220524 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:55:22.220524 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 26 14:55:22.220524 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 14:55:22.220524 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 14:55:22.220524 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 14:55:22.220524 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 14:55:22.220524 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 14:55:22.220524 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 14:55:22.220524 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 26 14:55:22.220524 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 26 14:55:22.224557 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 316/620] 10.222.0.46 () {56 vars in 1099 bytes} [Thu May 26 14:55:21 2022] GET /identity/v3/users/glance => generated 84 bytes in 342 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:55:22.231063 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8044a372-e604-4c68-94aa-2e22eb6f8a8c None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:22.338534 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6710e0b9-2daa-41d7-8b58-d5cd65685cdd None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:22.365916 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:22.388967 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8044a372-e604-4c68-94aa-2e22eb6f8a8c None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:22.410969 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ceb894a-9e53-4767-ae21-a7e8423fd22d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:22.412068 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ceb894a-9e53-4767-ae21-a7e8423fd22d None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:22.412215 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ceb894a-9e53-4767-ae21-a7e8423fd22d None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:22.412484 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ceb894a-9e53-4767-ae21-a7e8423fd22d None admin] PATH_INFO: `/v3/users/swift` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:22.431154 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ceb894a-9e53-4767-ae21-a7e8423fd22d None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:22.432248 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ceb894a-9e53-4767-ae21-a7e8423fd22d None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:22.441823 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:22.451759 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a652ae28-08d3-40db-a488-ac533f097754 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:22.453249 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a652ae28-08d3-40db-a488-ac533f097754 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:22.454194 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a652ae28-08d3-40db-a488-ac533f097754 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:22.454923 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a652ae28-08d3-40db-a488-ac533f097754 None admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:22.456619 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a652ae28-08d3-40db-a488-ac533f097754 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:55:22.457401 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a652ae28-08d3-40db-a488-ac533f097754 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:22.460225 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a652ae28-08d3-40db-a488-ac533f097754 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:22.470340 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-3ceb894a-9e53-4767-ae21-a7e8423fd22d None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 26 14:55:22.470340 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:55:22.470340 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:55:22.470340 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:55:22.470340 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:55:22.470340 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:55:22.470340 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:55:22.470340 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:55:22.470340 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:55:22.470340 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:55:22.470340 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:55:22.470340 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:55:22.470340 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 14:55:22.470340 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 14:55:22.470340 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 14:55:22.470340 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 14:55:22.471778 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:55:22.471778 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:55:22.471778 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 14:55:22.471778 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 14:55:22.471778 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 14:55:22.471778 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 14:55:22.471778 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:55:22.471778 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:55:22.471778 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:55:22.471778 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:55:22.471778 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:55:22.471778 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 26 14:55:22.471778 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:55:22.471778 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 26 14:55:22.471778 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:55:22.471778 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:55:22.471778 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:55:22.471778 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 26 14:55:22.473263 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:55:22.473263 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 26 14:55:22.473263 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 14:55:22.473263 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 14:55:22.473263 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 14:55:22.473263 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 14:55:22.473263 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 14:55:22.473263 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 14:55:22.473263 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 26 14:55:22.473263 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 26 14:55:22.473263 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 305/621] 10.222.0.46 () {56 vars in 1096 bytes} [Thu May 26 14:55:22 2022] GET /identity/v3/users/swift => generated 83 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 14:55:22.493652 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ceb894a-9e53-4767-ae21-a7e8423fd22d None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:22.502964 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 317/622] 10.222.0.46 () {56 vars in 1137 bytes} [Thu May 26 14:55:22 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 14:55:22.511425 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a652ae28-08d3-40db-a488-ac533f097754 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:22.675236 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ceb894a-9e53-4767-ae21-a7e8423fd22d None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:22.704805 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a652ae28-08d3-40db-a488-ac533f097754 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:22.738324 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e0da3c-7ea5-45d3-8607-9e68b6d6e97f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:22.739035 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e0da3c-7ea5-45d3-8607-9e68b6d6e97f None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:22.739331 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e0da3c-7ea5-45d3-8607-9e68b6d6e97f None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:22.739615 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e0da3c-7ea5-45d3-8607-9e68b6d6e97f None admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:22.740576 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e0da3c-7ea5-45d3-8607-9e68b6d6e97f None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:55:22.740894 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e0da3c-7ea5-45d3-8607-9e68b6d6e97f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:22.742409 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e0da3c-7ea5-45d3-8607-9e68b6d6e97f None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:22.745000 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742c5525-2218-4194-8e7a-7573c0e78053 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:22.746014 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742c5525-2218-4194-8e7a-7573c0e78053 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:22.746506 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742c5525-2218-4194-8e7a-7573c0e78053 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:22.747005 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742c5525-2218-4194-8e7a-7573c0e78053 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:22.774202 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 306/623] 10.222.0.46 () {56 vars in 1135 bytes} [Thu May 26 14:55:22 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:55:22.780377 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e0da3c-7ea5-45d3-8607-9e68b6d6e97f None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:22.822885 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742c5525-2218-4194-8e7a-7573c0e78053 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:22.824556 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742c5525-2218-4194-8e7a-7573c0e78053 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:22.854990 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:22.856758 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:22.931650 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e0da3c-7ea5-45d3-8607-9e68b6d6e97f None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:22.974774 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e48f13-6596-433e-a2c1-45e102262e09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:22.975608 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e48f13-6596-433e-a2c1-45e102262e09 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:22.975898 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e48f13-6596-433e-a2c1-45e102262e09 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:22.976233 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e48f13-6596-433e-a2c1-45e102262e09 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:22.989822 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e48f13-6596-433e-a2c1-45e102262e09 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:22.991588 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e48f13-6596-433e-a2c1-45e102262e09 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:23.012339 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-d9e48f13-6596-433e-a2c1-45e102262e09 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 14:55:23.012339 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:55:23.012339 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:55:23.012339 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:55:23.012339 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:55:23.012339 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:55:23.012339 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:55:23.012339 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:55:23.012339 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:55:23.012339 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:55:23.012339 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:55:23.012339 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:55:23.012339 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 14:55:23.012339 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 14:55:23.012339 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 14:55:23.013332 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 14:55:23.013332 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:55:23.013332 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:55:23.013332 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:55:23.013332 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:55:23.013332 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:55:23.013332 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:55:23.013332 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:55:23.013332 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 26 14:55:23.013332 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:55:23.013332 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 26 14:55:23.013332 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:55:23.013332 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:55:23.013332 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:55:23.013332 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 26 14:55:23.013332 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:55:23.013332 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 26 14:55:23.014702 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 14:55:23.014702 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 14:55:23.014702 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 14:55:23.014702 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 26 14:55:23.014702 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 307/624] 10.222.0.46 () {56 vars in 1108 bytes} [Thu May 26 14:55:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:55:23.019744 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e48f13-6596-433e-a2c1-45e102262e09 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:23.027975 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 318/625] 10.222.0.46 () {58 vars in 1282 bytes} [Thu May 26 14:55:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 517 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 26 14:55:23.033435 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742c5525-2218-4194-8e7a-7573c0e78053 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:23.141926 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:23.161980 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742c5525-2218-4194-8e7a-7573c0e78053 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:23.179900 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:23.189329 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e48f13-6596-433e-a2c1-45e102262e09 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:23.239974 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aeb2d79-b12a-46d9-9625-2933f33fb17c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:23.241210 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aeb2d79-b12a-46d9-9625-2933f33fb17c None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:23.241758 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aeb2d79-b12a-46d9-9625-2933f33fb17c None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:23.242259 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aeb2d79-b12a-46d9-9625-2933f33fb17c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:23.262572 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aeb2d79-b12a-46d9-9625-2933f33fb17c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:23.265395 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aeb2d79-b12a-46d9-9625-2933f33fb17c None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:23.278827 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3374c2-c6cf-4ab0-a03e-ec6da7e8184d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:23.279521 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3374c2-c6cf-4ab0-a03e-ec6da7e8184d None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:23.279802 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3374c2-c6cf-4ab0-a03e-ec6da7e8184d None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:23.280075 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3374c2-c6cf-4ab0-a03e-ec6da7e8184d None admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:23.280815 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3374c2-c6cf-4ab0-a03e-ec6da7e8184d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:55:23.281123 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3374c2-c6cf-4ab0-a03e-ec6da7e8184d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:23.282717 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3374c2-c6cf-4ab0-a03e-ec6da7e8184d None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:23.298575 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-2aeb2d79-b12a-46d9-9625-2933f33fb17c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 14:55:23.298575 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:55:23.298575 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:55:23.298575 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:55:23.298575 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:55:23.298575 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:55:23.298575 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:55:23.298575 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:55:23.298575 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:55:23.298575 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:55:23.298575 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:55:23.298575 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:55:23.298575 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 14:55:23.298575 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 14:55:23.298575 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 14:55:23.299513 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 14:55:23.299513 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:55:23.299513 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:55:23.299513 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:55:23.299513 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:55:23.299513 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:55:23.299513 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:55:23.299513 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:55:23.299513 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 26 14:55:23.299513 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:55:23.299513 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 26 14:55:23.299513 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:55:23.299513 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:55:23.299513 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:55:23.299513 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 26 14:55:23.299513 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:55:23.299513 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 26 14:55:23.300485 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 14:55:23.300485 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 14:55:23.300485 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 14:55:23.300485 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 26 14:55:23.300485 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 319/626] 10.222.0.46 () {56 vars in 1108 bytes} [Thu May 26 14:55:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 14:55:23.301409 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 308/627] 10.222.0.46 () {56 vars in 1109 bytes} [Thu May 26 14:55:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 14:55:23.306134 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839c0e71-44f3-47ee-bd4c-888e5d0c3558 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:23.306456 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839c0e71-44f3-47ee-bd4c-888e5d0c3558 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:23.306850 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839c0e71-44f3-47ee-bd4c-888e5d0c3558 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:23.308292 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3374c2-c6cf-4ab0-a03e-ec6da7e8184d None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:23.309956 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 320/628] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 14:55:23 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 14:55:23.311512 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839c0e71-44f3-47ee-bd4c-888e5d0c3558 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:23.430129 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3374c2-c6cf-4ab0-a03e-ec6da7e8184d None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:23.459693 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839c0e71-44f3-47ee-bd4c-888e5d0c3558 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:23.483473 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb8d5378-a66b-4b90-a15d-bb7b934df551 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:23.484761 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8d5378-a66b-4b90-a15d-bb7b934df551 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:23.485322 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8d5378-a66b-4b90-a15d-bb7b934df551 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:23.486036 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb8d5378-a66b-4b90-a15d-bb7b934df551 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:23.495591 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8d5378-a66b-4b90-a15d-bb7b934df551 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:23.497071 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb8d5378-a66b-4b90-a15d-bb7b934df551 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:23.502351 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e67821b-5c4a-4bc9-a8b0-f3d2048cc3d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:23.503518 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e67821b-5c4a-4bc9-a8b0-f3d2048cc3d3 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:23.503518 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e67821b-5c4a-4bc9-a8b0-f3d2048cc3d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:23.503518 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e67821b-5c4a-4bc9-a8b0-f3d2048cc3d3 None admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:23.504267 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e67821b-5c4a-4bc9-a8b0-f3d2048cc3d3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:55:23.504575 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e67821b-5c4a-4bc9-a8b0-f3d2048cc3d3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:23.506274 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e67821b-5c4a-4bc9-a8b0-f3d2048cc3d3 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:23.514689 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-bb8d5378-a66b-4b90-a15d-bb7b934df551 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 14:55:23.514689 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:55:23.514689 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:55:23.514689 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:55:23.514689 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:55:23.514689 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:55:23.514689 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:55:23.514689 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:55:23.514689 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:55:23.514689 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:55:23.514689 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:55:23.514689 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:55:23.514689 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 14:55:23.514689 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 14:55:23.514689 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 14:55:23.515645 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 14:55:23.515645 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:55:23.515645 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:55:23.515645 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:55:23.515645 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:55:23.515645 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:55:23.515645 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:55:23.515645 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:55:23.515645 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 26 14:55:23.515645 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:55:23.515645 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 26 14:55:23.515645 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:55:23.515645 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:55:23.515645 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:55:23.515645 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 26 14:55:23.515645 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:55:23.515645 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 26 14:55:23.516527 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 14:55:23.516527 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 14:55:23.516527 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 14:55:23.516527 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 14:55:23.516527 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 14:55:23.516527 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 14:55:23.516527 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 14:55:23.516527 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 26 14:55:23.516527 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 309/629] 10.222.0.46 () {56 vars in 1111 bytes} [Thu May 26 14:55:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:55:23.518029 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8136d7-08f7-42aa-8369-c76b0c668783 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:23.518029 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8136d7-08f7-42aa-8369-c76b0c668783 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:23.518278 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8136d7-08f7-42aa-8369-c76b0c668783 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:23.523606 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 321/630] 10.222.0.46 () {56 vars in 1109 bytes} [Thu May 26 14:55:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:55:23.524945 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb47644-86ae-49a4-ae28-b19b0b10ceed None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:23.525234 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb47644-86ae-49a4-ae28-b19b0b10ceed None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:23.525543 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb47644-86ae-49a4-ae28-b19b0b10ceed None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:23.529705 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 322/631] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 14:55:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 14:55:23.529705 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb47644-86ae-49a4-ae28-b19b0b10ceed None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:23.536216 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:23.546162 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-7b8136d7-08f7-42aa-8369-c76b0c668783 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:55:23.547369 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-7b8136d7-08f7-42aa-8369-c76b0c668783 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:55:23.549608 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:23.654700 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 310/632] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 14:55:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 14:55:23.656265 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8136d7-08f7-42aa-8369-c76b0c668783 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:23.710498 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb47644-86ae-49a4-ae28-b19b0b10ceed None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:23.764801 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e12c14-0abd-461b-ab05-bc7a4407c123 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:23.765972 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e12c14-0abd-461b-ab05-bc7a4407c123 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:23.766260 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e12c14-0abd-461b-ab05-bc7a4407c123 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:23.767474 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e12c14-0abd-461b-ab05-bc7a4407c123 None admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:23.768476 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e12c14-0abd-461b-ab05-bc7a4407c123 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:55:23.768789 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e12c14-0abd-461b-ab05-bc7a4407c123 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:23.770262 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e12c14-0abd-461b-ab05-bc7a4407c123 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:23.787311 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 323/633] 10.222.0.46 () {56 vars in 1148 bytes} [Thu May 26 14:55:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:55:23.788402 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c002601-06a8-4bc3-a855-dd3f342f67fd None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:23.788687 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c002601-06a8-4bc3-a855-dd3f342f67fd None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:23.788968 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c002601-06a8-4bc3-a855-dd3f342f67fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:23.807134 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-6c002601-06a8-4bc3-a855-dd3f342f67fd None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:55:23.808713 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-6c002601-06a8-4bc3-a855-dd3f342f67fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:55:23.817647 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8136d7-08f7-42aa-8369-c76b0c668783 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:23.859243 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d720f20-1eca-4bd8-8923-405f170d39a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:23.860043 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d720f20-1eca-4bd8-8923-405f170d39a3 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:23.860321 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d720f20-1eca-4bd8-8923-405f170d39a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:23.860594 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d720f20-1eca-4bd8-8923-405f170d39a3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:23.869135 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d720f20-1eca-4bd8-8923-405f170d39a3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:23.870653 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d720f20-1eca-4bd8-8923-405f170d39a3 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:23.873101 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:23.876749 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:23.893497 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-2d720f20-1eca-4bd8-8923-405f170d39a3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 14:55:23.893497 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:55:23.893497 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:55:23.893497 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:55:23.893497 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:55:23.893497 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:55:23.893497 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:55:23.893497 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:55:23.893497 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:55:23.893497 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:55:23.893497 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:55:23.893497 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:55:23.893497 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 14:55:23.893497 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 14:55:23.893497 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 14:55:23.894488 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 14:55:23.894488 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:55:23.894488 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:55:23.894488 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:55:23.894488 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:55:23.894488 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:55:23.894488 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:55:23.894488 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:55:23.894488 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 26 14:55:23.894488 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:55:23.894488 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 26 14:55:23.894488 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:55:23.894488 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:55:23.894488 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:55:23.894488 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 26 14:55:23.894488 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:55:23.894488 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 26 14:55:23.895361 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 14:55:23.895361 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 14:55:23.895361 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 14:55:23.895361 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 14:55:23.895361 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 14:55:23.895361 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 14:55:23.895361 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 14:55:23.895361 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 26 14:55:23.897615 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 311/634] 10.222.0.46 () {56 vars in 1111 bytes} [Thu May 26 14:55:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:55:23.898700 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d720f20-1eca-4bd8-8923-405f170d39a3 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:23.900435 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 324/635] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 14:55:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 14:55:23.902113 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c002601-06a8-4bc3-a855-dd3f342f67fd None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:24.050216 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c002601-06a8-4bc3-a855-dd3f342f67fd None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:24.066104 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d720f20-1eca-4bd8-8923-405f170d39a3 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:24.113407 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fefbc0-bb99-428b-a64a-657159ac2906 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:24.114776 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fefbc0-bb99-428b-a64a-657159ac2906 None admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:24.115105 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fefbc0-bb99-428b-a64a-657159ac2906 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:24.115673 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fefbc0-bb99-428b-a64a-657159ac2906 None admin] PATH_INFO: `/v3/projects/a17fc8227d4c4da6b2847b350e912ee3/users/a4bea272624b45d8ac2ad27bd07df885/roles/eb76213820fa4c4e9b0a5d5ac2a08b41` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:24.122425 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82096a27-04a4-4c2e-b0b9-60cee17255d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:24.123425 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82096a27-04a4-4c2e-b0b9-60cee17255d1 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:24.123830 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82096a27-04a4-4c2e-b0b9-60cee17255d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:24.124237 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82096a27-04a4-4c2e-b0b9-60cee17255d1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:24.138931 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82096a27-04a4-4c2e-b0b9-60cee17255d1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 14:55:24.141672 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fefbc0-bb99-428b-a64a-657159ac2906 None admin] RBAC: Authorizing `identity:create_grant(project_id=a17fc8227d4c4da6b2847b350e912ee3, user_id=a4bea272624b45d8ac2ad27bd07df885, role_id=eb76213820fa4c4e9b0a5d5ac2a08b41)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:24.144182 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fefbc0-bb99-428b-a64a-657159ac2906 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:24.148342 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-82096a27-04a4-4c2e-b0b9-60cee17255d1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 26 14:55:24.148342 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:55:24.148342 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:55:24.148342 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:55:24.148342 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:55:24.148342 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:55:24.148342 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:55:24.148342 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:55:24.148342 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:55:24.148342 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:55:24.148342 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:55:24.148342 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:55:24.148342 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 14:55:24.148342 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 14:55:24.148342 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 14:55:24.148342 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 14:55:24.149812 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 14:55:24.149812 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 14:55:24.149812 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 14:55:24.149812 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 14:55:24.149812 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:55:24.149812 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:55:24.149812 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:55:24.149812 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:55:24.149812 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:55:24.149812 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:55:24.149812 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:55:24.149812 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 26 14:55:24.149812 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:55:24.149812 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 26 14:55:24.149812 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:55:24.149812 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:55:24.150765 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:55:24.150765 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 26 14:55:24.150765 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:55:24.150765 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 26 14:55:24.150765 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 14:55:24.150765 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 14:55:24.150765 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 14:55:24.150765 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 14:55:24.150765 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 14:55:24.150765 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 14:55:24.150765 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 26 14:55:24.150765 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 26 14:55:24.150765 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 312/636] 10.222.0.46 () {56 vars in 1102 bytes} [Thu May 26 14:55:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:55:24.150765 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82096a27-04a4-4c2e-b0b9-60cee17255d1 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:24.172152 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 325/637] 10.222.0.46 () {58 vars in 1439 bytes} [Thu May 26 14:55:23 2022] PUT /identity/v3/projects/a17fc8227d4c4da6b2847b350e912ee3/users/a4bea272624b45d8ac2ad27bd07df885/roles/eb76213820fa4c4e9b0a5d5ac2a08b41 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 14:55:24.173012 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fefbc0-bb99-428b-a64a-657159ac2906 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:24.254539 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:24.260992 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:24.327663 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82096a27-04a4-4c2e-b0b9-60cee17255d1 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:24.356360 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fefbc0-bb99-428b-a64a-657159ac2906 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:24.387212 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66dc1f7e-80c5-4ef8-b88d-6e860e61d76d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:24.388662 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dc1f7e-80c5-4ef8-b88d-6e860e61d76d None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:24.389741 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dc1f7e-80c5-4ef8-b88d-6e860e61d76d None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:24.390202 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dc1f7e-80c5-4ef8-b88d-6e860e61d76d None admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:24.391619 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66dc1f7e-80c5-4ef8-b88d-6e860e61d76d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:55:24.392194 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66dc1f7e-80c5-4ef8-b88d-6e860e61d76d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:24.395025 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66dc1f7e-80c5-4ef8-b88d-6e860e61d76d None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:24.411702 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7c9614-0478-4110-b54d-325edf0b0497 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:24.413056 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7c9614-0478-4110-b54d-325edf0b0497 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:24.413540 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7c9614-0478-4110-b54d-325edf0b0497 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:24.413999 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7c9614-0478-4110-b54d-325edf0b0497 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:24.420240 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 313/638] 10.222.0.46 () {56 vars in 1148 bytes} [Thu May 26 14:55:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:55:24.421222 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66dc1f7e-80c5-4ef8-b88d-6e860e61d76d None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:24.431824 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7c9614-0478-4110-b54d-325edf0b0497 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 14:55:24.439975 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-de7c9614-0478-4110-b54d-325edf0b0497 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 26 14:55:24.439975 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:55:24.439975 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:55:24.439975 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:55:24.439975 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:55:24.439975 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:55:24.439975 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:55:24.439975 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:55:24.439975 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:55:24.439975 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:55:24.439975 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:55:24.439975 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:55:24.439975 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 14:55:24.439975 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 14:55:24.439975 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 14:55:24.439975 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 14:55:24.440971 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 14:55:24.440971 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 14:55:24.440971 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 14:55:24.440971 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 14:55:24.440971 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:55:24.440971 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:55:24.440971 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:55:24.440971 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:55:24.440971 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:55:24.440971 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:55:24.440971 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:55:24.440971 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 26 14:55:24.440971 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:55:24.440971 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 26 14:55:24.440971 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:55:24.440971 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:55:24.441920 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:55:24.441920 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 26 14:55:24.441920 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:55:24.441920 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 26 14:55:24.441920 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 14:55:24.441920 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 14:55:24.441920 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 14:55:24.441920 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 14:55:24.441920 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 14:55:24.441920 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 14:55:24.441920 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 26 14:55:24.441920 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 26 14:55:24.441920 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 326/639] 10.222.0.46 () {56 vars in 1102 bytes} [Thu May 26 14:55:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:55:24.442608 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cc2ff6-e896-4e90-bc68-2c49c13478d6 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:24.442961 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cc2ff6-e896-4e90-bc68-2c49c13478d6 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:24.443180 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cc2ff6-e896-4e90-bc68-2c49c13478d6 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:24.444791 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 327/640] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 14:55:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 14:55:24.445550 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cc2ff6-e896-4e90-bc68-2c49c13478d6 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:24.574442 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66dc1f7e-80c5-4ef8-b88d-6e860e61d76d None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:24.598821 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cc2ff6-e896-4e90-bc68-2c49c13478d6 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:24.601046 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:24.609079 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:24.636353 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cff088-9294-489e-b656-362618131b13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:24.637046 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cff088-9294-489e-b656-362618131b13 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:24.637325 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cff088-9294-489e-b656-362618131b13 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:24.637636 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cff088-9294-489e-b656-362618131b13 None admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:24.638388 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66cff088-9294-489e-b656-362618131b13 None admin] RBAC: Adding query filter params (name=service) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:55:24.638700 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66cff088-9294-489e-b656-362618131b13 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:24.640109 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66cff088-9294-489e-b656-362618131b13 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:24.647548 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd6a92a-9fb1-46cf-af35-734dadb91b70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:24.648208 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd6a92a-9fb1-46cf-af35-734dadb91b70 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:24.648493 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd6a92a-9fb1-46cf-af35-734dadb91b70 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:24.648771 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd6a92a-9fb1-46cf-af35-734dadb91b70 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:24.656060 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 314/641] 10.222.0.46 () {56 vars in 1103 bytes} [Thu May 26 14:55:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:55:24.656304 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd6a92a-9fb1-46cf-af35-734dadb91b70 None admin] RBAC: Adding query filter params (role.id=eb76213820fa4c4e9b0a5d5ac2a08b41, scope.project.id=a17fc8227d4c4da6b2847b350e912ee3, user.id=0a436dbdee624f44ad76cca8416b0ad1) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:55:24.656304 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd6a92a-9fb1-46cf-af35-734dadb91b70 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:24.656588 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c70a13-9149-4fca-a3bb-ef671829b9a9 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:24.656588 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c70a13-9149-4fca-a3bb-ef671829b9a9 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:24.656588 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13c70a13-9149-4fca-a3bb-ef671829b9a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:24.656950 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd6a92a-9fb1-46cf-af35-734dadb91b70 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:24.664065 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 328/642] 10.222.0.46 () {56 vars in 1374 bytes} [Thu May 26 14:55:24 2022] GET /identity/v3/role_assignments?user.id=0a436dbdee624f44ad76cca8416b0ad1&scope.project.id=a17fc8227d4c4da6b2847b350e912ee3&role.id=eb76213820fa4c4e9b0a5d5ac2a08b41 => generated 608 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:55:24.668353 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd6a92a-9fb1-46cf-af35-734dadb91b70 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:24.673604 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-13c70a13-9149-4fca-a3bb-ef671829b9a9 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:55:24.675322 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-13c70a13-9149-4fca-a3bb-ef671829b9a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:55:24.776289 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 315/643] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 14:55:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 14:55:24.778283 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c70a13-9149-4fca-a3bb-ef671829b9a9 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:24.819380 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd6a92a-9fb1-46cf-af35-734dadb91b70 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:24.867981 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bd8834-4e0b-4339-9f70-29553f7ea8a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:24.868846 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bd8834-4e0b-4339-9f70-29553f7ea8a8 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:24.869135 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bd8834-4e0b-4339-9f70-29553f7ea8a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:24.869412 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bd8834-4e0b-4339-9f70-29553f7ea8a8 None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:24.870250 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bd8834-4e0b-4339-9f70-29553f7ea8a8 None admin] RBAC: Adding query filter params (name=service) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:55:24.870567 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bd8834-4e0b-4339-9f70-29553f7ea8a8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:24.873058 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bd8834-4e0b-4339-9f70-29553f7ea8a8 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:24.885990 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 329/644] 10.222.0.46 () {56 vars in 1103 bytes} [Thu May 26 14:55:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:55:24.886892 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bd8834-4e0b-4339-9f70-29553f7ea8a8 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:24.933246 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:24.936809 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13c70a13-9149-4fca-a3bb-ef671829b9a9 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:24.961698 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:24.990009 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99d8dbb-54ab-4019-8f07-a1c7074b1c7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:24.990821 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99d8dbb-54ab-4019-8f07-a1c7074b1c7e None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:24.991104 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99d8dbb-54ab-4019-8f07-a1c7074b1c7e None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:24.991419 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99d8dbb-54ab-4019-8f07-a1c7074b1c7e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:25.005821 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99d8dbb-54ab-4019-8f07-a1c7074b1c7e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:25.008362 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99d8dbb-54ab-4019-8f07-a1c7074b1c7e None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:25.036328 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-c99d8dbb-54ab-4019-8f07-a1c7074b1c7e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 14:55:25.036328 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:55:25.036328 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:55:25.036328 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:55:25.036328 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:55:25.036328 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:55:25.036328 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:55:25.036328 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:55:25.036328 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:55:25.036328 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:55:25.036328 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:55:25.036328 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:55:25.036328 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 14:55:25.036328 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 14:55:25.036328 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 14:55:25.037986 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 14:55:25.037986 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:55:25.037986 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:55:25.037986 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:55:25.037986 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:55:25.037986 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:55:25.037986 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:55:25.037986 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:55:25.037986 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 26 14:55:25.037986 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:55:25.037986 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 26 14:55:25.037986 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:55:25.037986 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:55:25.037986 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:55:25.037986 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 26 14:55:25.037986 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:55:25.037986 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 26 14:55:25.039392 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 14:55:25.039392 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 14:55:25.039392 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 14:55:25.039392 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 26 14:55:25.039392 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 316/645] 10.222.0.46 () {56 vars in 1108 bytes} [Thu May 26 14:55:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:55:25.039392 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99d8dbb-54ab-4019-8f07-a1c7074b1c7e None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:25.049871 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bd8834-4e0b-4339-9f70-29553f7ea8a8 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:25.091014 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93c4c67-6719-41d5-8d74-657011212e11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:25.096576 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93c4c67-6719-41d5-8d74-657011212e11 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:25.096576 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93c4c67-6719-41d5-8d74-657011212e11 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:25.096576 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93c4c67-6719-41d5-8d74-657011212e11 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:25.110391 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93c4c67-6719-41d5-8d74-657011212e11 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 14:55:25.123505 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-c93c4c67-6719-41d5-8d74-657011212e11 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 26 14:55:25.123505 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:55:25.123505 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:55:25.123505 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:55:25.123505 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:55:25.123505 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:55:25.123505 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:55:25.123505 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:55:25.123505 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:55:25.123505 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:55:25.123505 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:55:25.123505 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:55:25.123505 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 14:55:25.123505 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 14:55:25.123505 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 14:55:25.123505 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 14:55:25.124947 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 14:55:25.124947 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 14:55:25.124947 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 14:55:25.124947 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 14:55:25.124947 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:55:25.124947 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:55:25.124947 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:55:25.124947 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:55:25.124947 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:55:25.124947 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:55:25.124947 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:55:25.124947 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 26 14:55:25.124947 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:55:25.124947 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 26 14:55:25.124947 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:55:25.124947 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:55:25.126534 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:55:25.126534 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 26 14:55:25.126534 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:55:25.126534 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 26 14:55:25.126534 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 14:55:25.126534 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 14:55:25.126534 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 14:55:25.126534 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 14:55:25.126534 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 14:55:25.126534 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 14:55:25.126534 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 26 14:55:25.126534 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 26 14:55:25.126534 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 330/646] 10.222.0.46 () {56 vars in 1102 bytes} [Thu May 26 14:55:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:55:25.127443 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93c4c67-6719-41d5-8d74-657011212e11 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:25.169410 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99d8dbb-54ab-4019-8f07-a1c7074b1c7e None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:25.212069 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dba0ce-90af-451e-8254-2ddeaa09b991 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:25.213044 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dba0ce-90af-451e-8254-2ddeaa09b991 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:25.213532 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dba0ce-90af-451e-8254-2ddeaa09b991 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:25.213964 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dba0ce-90af-451e-8254-2ddeaa09b991 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:25.226165 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96dba0ce-90af-451e-8254-2ddeaa09b991 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:25.227649 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96dba0ce-90af-451e-8254-2ddeaa09b991 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:25.248271 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-96dba0ce-90af-451e-8254-2ddeaa09b991 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 14:55:25.248271 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:55:25.248271 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:55:25.248271 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:55:25.248271 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:55:25.248271 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:55:25.248271 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:55:25.248271 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:55:25.248271 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:55:25.248271 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:55:25.248271 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:55:25.248271 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:55:25.248271 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 14:55:25.248271 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 14:55:25.248271 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 14:55:25.249221 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 14:55:25.249221 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:55:25.249221 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:55:25.249221 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:55:25.249221 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:55:25.249221 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:55:25.249221 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:55:25.249221 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:55:25.249221 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 26 14:55:25.249221 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:55:25.249221 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 26 14:55:25.249221 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:55:25.249221 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:55:25.249221 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:55:25.249221 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 26 14:55:25.249221 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:55:25.249221 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 26 14:55:25.250333 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 14:55:25.250333 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 14:55:25.250333 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 14:55:25.250333 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 26 14:55:25.254630 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 317/647] 10.222.0.46 () {56 vars in 1108 bytes} [Thu May 26 14:55:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:55:25.255967 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dba0ce-90af-451e-8254-2ddeaa09b991 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:25.271348 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:25.275625 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:25.291324 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93c4c67-6719-41d5-8d74-657011212e11 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:25.332681 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bdfc49-9273-4847-b6a2-c061c8ada65f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:25.333360 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bdfc49-9273-4847-b6a2-c061c8ada65f None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:25.333673 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bdfc49-9273-4847-b6a2-c061c8ada65f None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:25.333957 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bdfc49-9273-4847-b6a2-c061c8ada65f None admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:25.334690 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bdfc49-9273-4847-b6a2-c061c8ada65f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:55:25.335006 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bdfc49-9273-4847-b6a2-c061c8ada65f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:25.336381 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bdfc49-9273-4847-b6a2-c061c8ada65f None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:25.350581 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 331/648] 10.222.0.46 () {56 vars in 1109 bytes} [Thu May 26 14:55:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:55:25.351720 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bdfc49-9273-4847-b6a2-c061c8ada65f None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:25.388710 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dba0ce-90af-451e-8254-2ddeaa09b991 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:25.446618 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4de94a6-09d3-4d11-bba6-000042b39b86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:25.447599 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4de94a6-09d3-4d11-bba6-000042b39b86 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:25.448579 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4de94a6-09d3-4d11-bba6-000042b39b86 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:25.449394 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4de94a6-09d3-4d11-bba6-000042b39b86 None admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:25.450598 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4de94a6-09d3-4d11-bba6-000042b39b86 None admin] RBAC: Adding query filter params (name=service) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:55:25.451100 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4de94a6-09d3-4d11-bba6-000042b39b86 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:25.453258 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4de94a6-09d3-4d11-bba6-000042b39b86 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:25.473995 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 318/649] 10.222.0.46 () {56 vars in 1103 bytes} [Thu May 26 14:55:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:55:25.473995 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4de94a6-09d3-4d11-bba6-000042b39b86 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:25.484248 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bdfc49-9273-4847-b6a2-c061c8ada65f None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:25.534115 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe312fe-71ca-4924-8272-e489bd4324a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:25.535395 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe312fe-71ca-4924-8272-e489bd4324a2 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:25.536366 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe312fe-71ca-4924-8272-e489bd4324a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:25.536857 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe312fe-71ca-4924-8272-e489bd4324a2 None admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:25.537825 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe312fe-71ca-4924-8272-e489bd4324a2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:55:25.538347 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe312fe-71ca-4924-8272-e489bd4324a2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:25.540441 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe312fe-71ca-4924-8272-e489bd4324a2 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:25.559218 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 332/650] 10.222.0.46 () {56 vars in 1109 bytes} [Thu May 26 14:55:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:55:25.560355 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe312fe-71ca-4924-8272-e489bd4324a2 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:25.567660 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:25.576185 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:25.607227 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4de94a6-09d3-4d11-bba6-000042b39b86 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:25.655301 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2049009d-28b4-4dbd-9c91-98aa6cfbbd39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:25.656031 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2049009d-28b4-4dbd-9c91-98aa6cfbbd39 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:25.656319 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2049009d-28b4-4dbd-9c91-98aa6cfbbd39 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:25.656600 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2049009d-28b4-4dbd-9c91-98aa6cfbbd39 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:25.665555 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2049009d-28b4-4dbd-9c91-98aa6cfbbd39 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:25.667070 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2049009d-28b4-4dbd-9c91-98aa6cfbbd39 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:25.682753 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-2049009d-28b4-4dbd-9c91-98aa6cfbbd39 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 26 14:55:25.682753 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:55:25.682753 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:55:25.682753 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:55:25.682753 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:55:25.682753 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:55:25.682753 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:55:25.682753 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:55:25.682753 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:55:25.682753 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:55:25.682753 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:55:25.682753 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:55:25.682753 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 14:55:25.682753 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 14:55:25.682753 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 14:55:25.682753 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 14:55:25.683726 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:55:25.683726 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:55:25.683726 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 14:55:25.683726 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 14:55:25.683726 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 14:55:25.683726 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 14:55:25.683726 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:55:25.683726 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:55:25.683726 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:55:25.683726 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:55:25.683726 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:55:25.683726 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 26 14:55:25.683726 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:55:25.683726 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 26 14:55:25.683726 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:55:25.683726 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:55:25.683726 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:55:25.683726 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 26 14:55:25.684663 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:55:25.684663 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 26 14:55:25.684663 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 14:55:25.684663 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 14:55:25.684663 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 14:55:25.684663 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 14:55:25.684663 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 14:55:25.684663 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 14:55:25.684663 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 26 14:55:25.684663 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 26 14:55:25.684663 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 319/651] 10.222.0.46 () {56 vars in 1093 bytes} [Thu May 26 14:55:25 2022] GET /identity/v3/users/nova => generated 82 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:55:25.685254 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2049009d-28b4-4dbd-9c91-98aa6cfbbd39 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:25.701286 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe312fe-71ca-4924-8272-e489bd4324a2 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:25.748936 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd28d2d-88d8-4784-83e9-c58ee4ff979b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:25.750016 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd28d2d-88d8-4784-83e9-c58ee4ff979b None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:25.750983 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd28d2d-88d8-4784-83e9-c58ee4ff979b None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:25.751426 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd28d2d-88d8-4784-83e9-c58ee4ff979b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:25.768027 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd28d2d-88d8-4784-83e9-c58ee4ff979b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:25.769997 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd28d2d-88d8-4784-83e9-c58ee4ff979b None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:25.793267 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-6fd28d2d-88d8-4784-83e9-c58ee4ff979b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 14:55:25.793267 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:55:25.793267 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:55:25.793267 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:55:25.793267 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:55:25.793267 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:55:25.793267 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:55:25.793267 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:55:25.793267 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:55:25.793267 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:55:25.793267 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:55:25.793267 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:55:25.793267 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 14:55:25.793267 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 14:55:25.793267 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 14:55:25.794354 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 14:55:25.794354 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:55:25.794354 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:55:25.794354 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:55:25.794354 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:55:25.794354 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:55:25.794354 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:55:25.794354 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:55:25.794354 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 26 14:55:25.794354 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:55:25.794354 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 26 14:55:25.794354 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:55:25.794354 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:55:25.794354 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:55:25.794354 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 26 14:55:25.794354 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:55:25.794354 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 26 14:55:25.795426 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 14:55:25.795426 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 14:55:25.795426 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 14:55:25.795426 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 26 14:55:25.802609 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 333/652] 10.222.0.46 () {56 vars in 1108 bytes} [Thu May 26 14:55:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:55:25.802609 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd28d2d-88d8-4784-83e9-c58ee4ff979b None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:25.812457 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2049009d-28b4-4dbd-9c91-98aa6cfbbd39 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:25.858392 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8e2925-b375-4331-88ca-e4c9190f9c58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:25.859917 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8e2925-b375-4331-88ca-e4c9190f9c58 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:25.860423 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8e2925-b375-4331-88ca-e4c9190f9c58 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:25.860911 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8e2925-b375-4331-88ca-e4c9190f9c58 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:25.864729 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:25.872232 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:25.875541 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8e2925-b375-4331-88ca-e4c9190f9c58 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:25.877017 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8e2925-b375-4331-88ca-e4c9190f9c58 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:25.899478 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-5c8e2925-b375-4331-88ca-e4c9190f9c58 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 26 14:55:25.899478 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:55:25.899478 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:55:25.899478 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:55:25.899478 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:55:25.899478 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:55:25.899478 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:55:25.899478 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:55:25.899478 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:55:25.899478 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:55:25.899478 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:55:25.899478 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:55:25.899478 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 14:55:25.899478 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 14:55:25.899478 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 14:55:25.899478 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 14:55:25.900476 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:55:25.900476 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:55:25.900476 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 14:55:25.900476 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 14:55:25.900476 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 14:55:25.900476 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 14:55:25.900476 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:55:25.900476 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:55:25.900476 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:55:25.900476 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:55:25.900476 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:55:25.900476 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 26 14:55:25.900476 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:55:25.900476 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 26 14:55:25.900476 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:55:25.900476 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:55:25.900476 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:55:25.900476 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 26 14:55:25.901472 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:55:25.901472 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 26 14:55:25.901472 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 14:55:25.901472 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 14:55:25.901472 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 14:55:25.901472 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 14:55:25.901472 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 14:55:25.901472 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 14:55:25.901472 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 26 14:55:25.901472 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 26 14:55:25.901472 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 320/653] 10.222.0.46 () {56 vars in 1099 bytes} [Thu May 26 14:55:25 2022] GET /identity/v3/users/cinder => generated 84 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:55:25.902078 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8e2925-b375-4331-88ca-e4c9190f9c58 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:25.934908 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd28d2d-88d8-4784-83e9-c58ee4ff979b None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:25.981199 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1f1283-cadf-4d64-b6f5-c91190dd6cf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:25.982548 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1f1283-cadf-4d64-b6f5-c91190dd6cf5 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:25.983052 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1f1283-cadf-4d64-b6f5-c91190dd6cf5 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:25.983537 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1f1283-cadf-4d64-b6f5-c91190dd6cf5 None admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:25.985654 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1f1283-cadf-4d64-b6f5-c91190dd6cf5 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:55:25.986250 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1f1283-cadf-4d64-b6f5-c91190dd6cf5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:25.988240 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1f1283-cadf-4d64-b6f5-c91190dd6cf5 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:26.016980 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 334/654] 10.222.0.46 () {56 vars in 1133 bytes} [Thu May 26 14:55:25 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:55:26.018815 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1f1283-cadf-4d64-b6f5-c91190dd6cf5 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:26.024760 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8e2925-b375-4331-88ca-e4c9190f9c58 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:26.066356 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed1bd1f-ac7d-4f45-be11-c34f7b54eb15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:26.067147 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed1bd1f-ac7d-4f45-be11-c34f7b54eb15 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:26.067507 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed1bd1f-ac7d-4f45-be11-c34f7b54eb15 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:26.067858 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed1bd1f-ac7d-4f45-be11-c34f7b54eb15 None admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:26.068658 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed1bd1f-ac7d-4f45-be11-c34f7b54eb15 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:55:26.069039 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed1bd1f-ac7d-4f45-be11-c34f7b54eb15 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:26.070593 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed1bd1f-ac7d-4f45-be11-c34f7b54eb15 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:26.089621 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 321/655] 10.222.0.46 () {56 vars in 1109 bytes} [Thu May 26 14:55:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:55:26.089621 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed1bd1f-ac7d-4f45-be11-c34f7b54eb15 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:26.160929 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:26.168446 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1f1283-cadf-4d64-b6f5-c91190dd6cf5 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:26.191062 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:26.210275 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4547e7-912d-4b99-846d-3ed74cb4d7d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:26.211322 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4547e7-912d-4b99-846d-3ed74cb4d7d4 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:26.211433 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4547e7-912d-4b99-846d-3ed74cb4d7d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:26.211685 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4547e7-912d-4b99-846d-3ed74cb4d7d4 None admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:26.212785 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4547e7-912d-4b99-846d-3ed74cb4d7d4 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:55:26.213101 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4547e7-912d-4b99-846d-3ed74cb4d7d4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:26.214812 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4547e7-912d-4b99-846d-3ed74cb4d7d4 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:26.223131 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed1bd1f-ac7d-4f45-be11-c34f7b54eb15 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:26.234816 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 335/656] 10.222.0.46 () {56 vars in 1137 bytes} [Thu May 26 14:55:26 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:55:26.236097 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4547e7-912d-4b99-846d-3ed74cb4d7d4 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:26.269067 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b3e817-1677-4354-8556-6ea7ee6b5f1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:26.269595 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b3e817-1677-4354-8556-6ea7ee6b5f1d None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:26.269885 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b3e817-1677-4354-8556-6ea7ee6b5f1d None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:26.270240 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b3e817-1677-4354-8556-6ea7ee6b5f1d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:26.350190 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b3e817-1677-4354-8556-6ea7ee6b5f1d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:26.351815 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b3e817-1677-4354-8556-6ea7ee6b5f1d None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:26.374397 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4547e7-912d-4b99-846d-3ed74cb4d7d4 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:26.415349 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:26.422799 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0e42df-dfd9-418e-a4d5-481c85ecced1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:26.423634 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0e42df-dfd9-418e-a4d5-481c85ecced1 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:26.423920 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0e42df-dfd9-418e-a4d5-481c85ecced1 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:26.424199 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0e42df-dfd9-418e-a4d5-481c85ecced1 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:26.433826 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0e42df-dfd9-418e-a4d5-481c85ecced1 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:26.434962 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0e42df-dfd9-418e-a4d5-481c85ecced1 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:26.451640 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-ae0e42df-dfd9-418e-a4d5-481c85ecced1 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 26 14:55:26.451640 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:55:26.451640 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:55:26.451640 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:55:26.451640 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:55:26.451640 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:55:26.451640 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:55:26.451640 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:55:26.451640 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:55:26.451640 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:55:26.451640 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:55:26.451640 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:55:26.451640 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 14:55:26.451640 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 14:55:26.451640 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 14:55:26.451640 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 14:55:26.452620 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:55:26.452620 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:55:26.452620 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 14:55:26.452620 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 14:55:26.452620 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 14:55:26.452620 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 14:55:26.452620 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:55:26.452620 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:55:26.452620 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:55:26.452620 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:55:26.452620 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:55:26.452620 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 26 14:55:26.452620 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:55:26.452620 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 26 14:55:26.452620 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:55:26.452620 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:55:26.452620 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:55:26.452620 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 26 14:55:26.453577 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:55:26.453577 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 26 14:55:26.453577 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 14:55:26.453577 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 14:55:26.453577 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 14:55:26.453577 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 14:55:26.453577 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 14:55:26.453577 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 14:55:26.453577 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 26 14:55:26.453577 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 26 14:55:26.457688 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 336/657] 10.222.0.46 () {56 vars in 1102 bytes} [Thu May 26 14:55:26 2022] GET /identity/v3/users/neutron => generated 85 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:55:26.458654 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0e42df-dfd9-418e-a4d5-481c85ecced1 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:26.482553 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 322/658] 10.222.0.46 () {58 vars in 1282 bytes} [Thu May 26 14:55:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 14:55:26.484334 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b3e817-1677-4354-8556-6ea7ee6b5f1d None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:26.501786 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:26.595422 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0e42df-dfd9-418e-a4d5-481c85ecced1 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:26.613146 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b3e817-1677-4354-8556-6ea7ee6b5f1d None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:26.644519 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a10b180-f4fe-4b8c-90c7-e9e6547a614f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:26.645168 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a10b180-f4fe-4b8c-90c7-e9e6547a614f None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:26.645467 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a10b180-f4fe-4b8c-90c7-e9e6547a614f None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:26.645753 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a10b180-f4fe-4b8c-90c7-e9e6547a614f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:26.655283 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43251b33-4a76-4601-ae19-1dfaf19b2826 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:26.655966 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43251b33-4a76-4601-ae19-1dfaf19b2826 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:26.656250 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43251b33-4a76-4601-ae19-1dfaf19b2826 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:26.656528 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43251b33-4a76-4601-ae19-1dfaf19b2826 None admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:26.657371 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43251b33-4a76-4601-ae19-1dfaf19b2826 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:55:26.657712 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43251b33-4a76-4601-ae19-1dfaf19b2826 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:26.659402 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43251b33-4a76-4601-ae19-1dfaf19b2826 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:26.682623 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 323/659] 10.222.0.46 () {56 vars in 1139 bytes} [Thu May 26 14:55:26 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:55:26.682623 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43251b33-4a76-4601-ae19-1dfaf19b2826 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:26.699756 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:26.726904 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a10b180-f4fe-4b8c-90c7-e9e6547a614f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:26.728376 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a10b180-f4fe-4b8c-90c7-e9e6547a614f None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:26.766729 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:26.795455 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43251b33-4a76-4601-ae19-1dfaf19b2826 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:26.832966 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff39e683-1d6b-484e-8195-2f2e0abe7ae3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:26.833803 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff39e683-1d6b-484e-8195-2f2e0abe7ae3 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:26.834084 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff39e683-1d6b-484e-8195-2f2e0abe7ae3 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:26.834380 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff39e683-1d6b-484e-8195-2f2e0abe7ae3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:26.842393 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 337/660] 10.222.0.46 () {58 vars in 1282 bytes} [Thu May 26 14:55:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 14:55:26.842393 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfa2d91-d91c-4d12-bde2-5d4b08e28392 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:26.842393 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfa2d91-d91c-4d12-bde2-5d4b08e28392 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:26.842393 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bfa2d91-d91c-4d12-bde2-5d4b08e28392 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:26.842393 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 338/661] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 14:55:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 14:55:26.843667 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfa2d91-d91c-4d12-bde2-5d4b08e28392 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:26.851011 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff39e683-1d6b-484e-8195-2f2e0abe7ae3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:26.852472 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff39e683-1d6b-484e-8195-2f2e0abe7ae3 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:26.871958 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-ff39e683-1d6b-484e-8195-2f2e0abe7ae3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 14:55:26.871958 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:55:26.871958 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:55:26.871958 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:55:26.871958 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:55:26.871958 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:55:26.871958 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:55:26.871958 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:55:26.871958 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:55:26.871958 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:55:26.871958 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:55:26.871958 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:55:26.871958 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 14:55:26.871958 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 14:55:26.871958 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 14:55:26.872959 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 14:55:26.872959 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:55:26.872959 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:55:26.872959 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:55:26.872959 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:55:26.872959 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:55:26.872959 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:55:26.872959 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:55:26.872959 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 26 14:55:26.872959 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:55:26.872959 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 26 14:55:26.872959 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:55:26.872959 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:55:26.872959 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:55:26.872959 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 26 14:55:26.872959 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:55:26.872959 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 26 14:55:26.873920 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 14:55:26.873920 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 14:55:26.873920 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 14:55:26.873920 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 26 14:55:26.873920 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 324/662] 10.222.0.46 () {56 vars in 1108 bytes} [Thu May 26 14:55:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:55:26.874903 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff39e683-1d6b-484e-8195-2f2e0abe7ae3 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:26.949089 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:26.968112 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bfa2d91-d91c-4d12-bde2-5d4b08e28392 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:26.989508 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff39e683-1d6b-484e-8195-2f2e0abe7ae3 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:27.009591 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6147f305-566b-41dc-91d5-ccd74f3a373b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:27.010818 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6147f305-566b-41dc-91d5-ccd74f3a373b None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:27.011123 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6147f305-566b-41dc-91d5-ccd74f3a373b None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:27.011405 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6147f305-566b-41dc-91d5-ccd74f3a373b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:27.016039 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:27.034428 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ffd25a-2617-43c7-a5f1-fb902800995e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:27.035392 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ffd25a-2617-43c7-a5f1-fb902800995e None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:27.035604 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ffd25a-2617-43c7-a5f1-fb902800995e None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:27.035782 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ffd25a-2617-43c7-a5f1-fb902800995e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:27.047524 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ffd25a-2617-43c7-a5f1-fb902800995e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:27.049005 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ffd25a-2617-43c7-a5f1-fb902800995e None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:27.069864 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-78ffd25a-2617-43c7-a5f1-fb902800995e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 14:55:27.069864 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:55:27.069864 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:55:27.069864 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:55:27.069864 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:55:27.069864 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:55:27.069864 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:55:27.069864 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:55:27.069864 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:55:27.069864 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:55:27.069864 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:55:27.069864 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:55:27.069864 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 14:55:27.069864 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 14:55:27.069864 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 14:55:27.070891 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 14:55:27.070891 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:55:27.070891 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:55:27.070891 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:55:27.070891 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:55:27.070891 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:55:27.070891 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:55:27.070891 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:55:27.070891 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 26 14:55:27.070891 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:55:27.070891 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 26 14:55:27.070891 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:55:27.070891 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:55:27.070891 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:55:27.070891 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 26 14:55:27.070891 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:55:27.070891 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 26 14:55:27.071784 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 14:55:27.071784 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 14:55:27.071784 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 14:55:27.071784 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 26 14:55:27.071784 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 325/663] 10.222.0.46 () {56 vars in 1108 bytes} [Thu May 26 14:55:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:55:27.072505 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1a55f0-965e-4e1d-a1d2-69a9ad06a8c9 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:27.073487 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1a55f0-965e-4e1d-a1d2-69a9ad06a8c9 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:27.073606 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1a55f0-965e-4e1d-a1d2-69a9ad06a8c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:27.092267 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-ad1a55f0-965e-4e1d-a1d2-69a9ad06a8c9 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:55:27.093509 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-ad1a55f0-965e-4e1d-a1d2-69a9ad06a8c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:55:27.101505 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6147f305-566b-41dc-91d5-ccd74f3a373b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:27.103490 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6147f305-566b-41dc-91d5-ccd74f3a373b None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:27.168869 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 326/664] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 14:55:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 14:55:27.168869 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1a55f0-965e-4e1d-a1d2-69a9ad06a8c9 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:27.233474 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 339/665] 10.222.0.46 () {58 vars in 1282 bytes} [Thu May 26 14:55:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 14:55:27.234605 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6147f305-566b-41dc-91d5-ccd74f3a373b None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:27.261267 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:27.288193 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:27.330320 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1a55f0-965e-4e1d-a1d2-69a9ad06a8c9 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:27.365831 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6147f305-566b-41dc-91d5-ccd74f3a373b None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:27.372641 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc631cde-334b-4980-860c-5b4da7980b87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:27.373541 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc631cde-334b-4980-860c-5b4da7980b87 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:27.373658 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc631cde-334b-4980-860c-5b4da7980b87 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:27.373924 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc631cde-334b-4980-860c-5b4da7980b87 None admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:27.374655 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc631cde-334b-4980-860c-5b4da7980b87 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:55:27.374957 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc631cde-334b-4980-860c-5b4da7980b87 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:27.376357 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc631cde-334b-4980-860c-5b4da7980b87 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:27.392090 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 327/666] 10.222.0.46 () {56 vars in 1109 bytes} [Thu May 26 14:55:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:55:27.392866 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc631cde-334b-4980-860c-5b4da7980b87 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:27.409986 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e55933-e4dc-483e-bb97-71dcc5b0e02d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:27.410734 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e55933-e4dc-483e-bb97-71dcc5b0e02d None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:27.411033 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e55933-e4dc-483e-bb97-71dcc5b0e02d None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:27.411317 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e55933-e4dc-483e-bb97-71dcc5b0e02d None admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:27.412197 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e55933-e4dc-483e-bb97-71dcc5b0e02d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:55:27.412576 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e55933-e4dc-483e-bb97-71dcc5b0e02d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:27.414035 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e55933-e4dc-483e-bb97-71dcc5b0e02d None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:27.430902 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 340/667] 10.222.0.46 () {56 vars in 1109 bytes} [Thu May 26 14:55:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:55:27.430902 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3390f27-c31f-4d36-8fbb-6c3474f8f926 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:27.430902 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3390f27-c31f-4d36-8fbb-6c3474f8f926 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:27.430902 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3390f27-c31f-4d36-8fbb-6c3474f8f926 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:27.438758 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 341/668] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 14:55:27 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 14:55:27.438758 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3390f27-c31f-4d36-8fbb-6c3474f8f926 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:27.506045 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc631cde-334b-4980-860c-5b4da7980b87 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:27.516023 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:27.540046 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:27.544760 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7059e486-4e5c-4e90-8cab-ae352157d5a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:27.545591 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7059e486-4e5c-4e90-8cab-ae352157d5a2 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:27.545878 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7059e486-4e5c-4e90-8cab-ae352157d5a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:27.546153 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7059e486-4e5c-4e90-8cab-ae352157d5a2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:27.547859 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3390f27-c31f-4d36-8fbb-6c3474f8f926 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:27.557719 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7059e486-4e5c-4e90-8cab-ae352157d5a2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 14:55:27.565637 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-7059e486-4e5c-4e90-8cab-ae352157d5a2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 26 14:55:27.565637 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:55:27.565637 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:55:27.565637 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:55:27.565637 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:55:27.565637 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:55:27.565637 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:55:27.565637 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:55:27.565637 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:55:27.565637 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:55:27.565637 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:55:27.565637 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:55:27.565637 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 14:55:27.565637 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 14:55:27.565637 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 14:55:27.565637 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 14:55:27.567846 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 14:55:27.567846 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 14:55:27.567846 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 14:55:27.567846 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 14:55:27.567846 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:55:27.567846 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:55:27.567846 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:55:27.567846 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:55:27.567846 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:55:27.567846 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:55:27.567846 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:55:27.567846 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 26 14:55:27.567846 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:55:27.567846 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 26 14:55:27.567846 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:55:27.567846 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:55:27.569227 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:55:27.569227 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 26 14:55:27.569227 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:55:27.569227 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 26 14:55:27.569227 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 14:55:27.569227 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 14:55:27.569227 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 14:55:27.569227 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 14:55:27.569227 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 14:55:27.569227 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 14:55:27.569227 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 26 14:55:27.569227 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 26 14:55:27.569227 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 328/669] 10.222.0.46 () {56 vars in 1102 bytes} [Thu May 26 14:55:27 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:55:27.569227 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7059e486-4e5c-4e90-8cab-ae352157d5a2 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:27.594899 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43054da3-a967-42ce-8bdc-923c45975045 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:27.595706 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43054da3-a967-42ce-8bdc-923c45975045 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:27.595986 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43054da3-a967-42ce-8bdc-923c45975045 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:27.596567 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43054da3-a967-42ce-8bdc-923c45975045 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:27.609652 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43054da3-a967-42ce-8bdc-923c45975045 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:27.611127 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43054da3-a967-42ce-8bdc-923c45975045 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:27.630224 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-43054da3-a967-42ce-8bdc-923c45975045 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 14:55:27.630224 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:55:27.630224 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:55:27.630224 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:55:27.630224 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:55:27.630224 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:55:27.630224 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:55:27.630224 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:55:27.630224 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:55:27.630224 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:55:27.630224 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:55:27.630224 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:55:27.630224 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 14:55:27.630224 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 14:55:27.630224 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 14:55:27.631223 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 14:55:27.631223 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:55:27.631223 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:55:27.631223 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:55:27.631223 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:55:27.631223 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:55:27.631223 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:55:27.631223 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:55:27.631223 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 26 14:55:27.631223 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:55:27.631223 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 26 14:55:27.631223 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:55:27.631223 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:55:27.631223 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:55:27.631223 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 26 14:55:27.631223 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:55:27.631223 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 26 14:55:27.632123 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 14:55:27.632123 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 14:55:27.632123 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 14:55:27.632123 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 26 14:55:27.632123 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 342/670] 10.222.0.46 () {56 vars in 1108 bytes} [Thu May 26 14:55:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:55:27.632407 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43054da3-a967-42ce-8bdc-923c45975045 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:27.695774 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7059e486-4e5c-4e90-8cab-ae352157d5a2 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:27.734489 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb52296-3fd7-4383-8e3f-e5b1a33bedfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:27.735299 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb52296-3fd7-4383-8e3f-e5b1a33bedfe None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:27.735576 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb52296-3fd7-4383-8e3f-e5b1a33bedfe None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:27.735865 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb52296-3fd7-4383-8e3f-e5b1a33bedfe None admin] PATH_INFO: `/v3/projects/service` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:27.743665 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb52296-3fd7-4383-8e3f-e5b1a33bedfe None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:27.745750 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb52296-3fd7-4383-8e3f-e5b1a33bedfe None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:27.747102 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43054da3-a967-42ce-8bdc-923c45975045 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:27.761777 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-ccb52296-3fd7-4383-8e3f-e5b1a33bedfe None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 14:55:27.761777 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:55:27.761777 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:55:27.761777 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:55:27.761777 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:55:27.761777 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:55:27.761777 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:55:27.761777 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:55:27.761777 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:55:27.761777 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:55:27.761777 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:55:27.761777 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:55:27.761777 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 14:55:27.761777 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 14:55:27.761777 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 14:55:27.762918 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 14:55:27.762918 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:55:27.762918 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:55:27.762918 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:55:27.762918 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:55:27.762918 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:55:27.762918 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:55:27.762918 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:55:27.762918 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 26 14:55:27.762918 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:55:27.762918 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 26 14:55:27.762918 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:55:27.762918 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:55:27.762918 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:55:27.762918 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 26 14:55:27.762918 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:55:27.762918 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 26 14:55:27.763995 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 14:55:27.763995 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 14:55:27.763995 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 14:55:27.763995 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 14:55:27.763995 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 14:55:27.763995 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 14:55:27.763995 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 14:55:27.763995 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 26 14:55:27.763995 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 329/671] 10.222.0.46 () {56 vars in 1111 bytes} [Thu May 26 14:55:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:55:27.764761 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77926ac-a203-4a60-84c2-780dd01c3dd0 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:27.764761 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77926ac-a203-4a60-84c2-780dd01c3dd0 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:27.764949 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77926ac-a203-4a60-84c2-780dd01c3dd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:27.787076 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-d77926ac-a203-4a60-84c2-780dd01c3dd0 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:55:27.788495 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-d77926ac-a203-4a60-84c2-780dd01c3dd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:55:27.793191 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf973458-dc24-454e-aa33-db6189b04779 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:27.794097 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf973458-dc24-454e-aa33-db6189b04779 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:27.794384 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf973458-dc24-454e-aa33-db6189b04779 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:27.794709 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf973458-dc24-454e-aa33-db6189b04779 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:27.802666 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf973458-dc24-454e-aa33-db6189b04779 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:27.804427 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf973458-dc24-454e-aa33-db6189b04779 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:27.814461 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:27.819747 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-bf973458-dc24-454e-aa33-db6189b04779 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 14:55:27.819747 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:55:27.819747 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:55:27.819747 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:55:27.819747 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:55:27.819747 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:55:27.819747 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:55:27.819747 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:55:27.819747 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:55:27.819747 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:55:27.819747 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:55:27.819747 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:55:27.819747 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 14:55:27.819747 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 14:55:27.819747 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 14:55:27.820747 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 14:55:27.820747 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:55:27.820747 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:55:27.820747 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:55:27.820747 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:55:27.820747 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:55:27.820747 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:55:27.820747 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:55:27.820747 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 26 14:55:27.820747 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:55:27.820747 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 26 14:55:27.820747 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:55:27.820747 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:55:27.820747 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:55:27.820747 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 26 14:55:27.820747 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:55:27.820747 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 26 14:55:27.821642 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 14:55:27.821642 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 14:55:27.821642 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 14:55:27.821642 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 14:55:27.821642 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 14:55:27.821642 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 14:55:27.821642 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 14:55:27.821642 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 26 14:55:27.821642 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 343/672] 10.222.0.46 () {56 vars in 1111 bytes} [Thu May 26 14:55:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:55:27.822144 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf973458-dc24-454e-aa33-db6189b04779 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:27.840167 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:27.865928 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 330/673] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 14:55:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 14:55:27.867015 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77926ac-a203-4a60-84c2-780dd01c3dd0 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:27.942903 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf973458-dc24-454e-aa33-db6189b04779 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:27.978525 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77926ac-a203-4a60-84c2-780dd01c3dd0 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:27.981869 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ac5c46-2387-40cf-9892-fc666f045d91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:27.982543 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ac5c46-2387-40cf-9892-fc666f045d91 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:27.982826 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ac5c46-2387-40cf-9892-fc666f045d91 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:27.983102 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ac5c46-2387-40cf-9892-fc666f045d91 None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:27.983859 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ac5c46-2387-40cf-9892-fc666f045d91 None admin] RBAC: Adding query filter params (name=service) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:55:27.984164 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ac5c46-2387-40cf-9892-fc666f045d91 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:27.985556 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ac5c46-2387-40cf-9892-fc666f045d91 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:27.994542 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 344/674] 10.222.0.46 () {56 vars in 1103 bytes} [Thu May 26 14:55:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:55:27.996517 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ac5c46-2387-40cf-9892-fc666f045d91 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:28.022989 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e1cdb0-2f96-4919-adb1-b99803884a0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:28.023658 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e1cdb0-2f96-4919-adb1-b99803884a0a None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:28.023939 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e1cdb0-2f96-4919-adb1-b99803884a0a None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:28.024211 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e1cdb0-2f96-4919-adb1-b99803884a0a None admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:28.024930 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e1cdb0-2f96-4919-adb1-b99803884a0a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:55:28.025233 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e1cdb0-2f96-4919-adb1-b99803884a0a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:28.026677 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e1cdb0-2f96-4919-adb1-b99803884a0a None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:28.038374 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 331/675] 10.222.0.46 () {56 vars in 1109 bytes} [Thu May 26 14:55:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:55:28.039415 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e1cdb0-2f96-4919-adb1-b99803884a0a None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:28.066401 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:28.071350 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:28.104615 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ac5c46-2387-40cf-9892-fc666f045d91 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:28.142017 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee0fdec-d7ce-483d-85a1-40bff0c575ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:28.142694 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee0fdec-d7ce-483d-85a1-40bff0c575ac None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:28.143419 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee0fdec-d7ce-483d-85a1-40bff0c575ac None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:28.143701 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee0fdec-d7ce-483d-85a1-40bff0c575ac None admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:28.144743 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee0fdec-d7ce-483d-85a1-40bff0c575ac None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:55:28.144880 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee0fdec-d7ce-483d-85a1-40bff0c575ac None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:28.146362 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee0fdec-d7ce-483d-85a1-40bff0c575ac None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:28.160596 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e1cdb0-2f96-4919-adb1-b99803884a0a None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:28.160812 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 345/676] 10.222.0.46 () {56 vars in 1148 bytes} [Thu May 26 14:55:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:55:28.161565 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee0fdec-d7ce-483d-85a1-40bff0c575ac None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:28.199302 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ddc6d2-5633-4b48-bc8c-4fdd0af3662e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:28.199970 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ddc6d2-5633-4b48-bc8c-4fdd0af3662e None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:28.200251 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ddc6d2-5633-4b48-bc8c-4fdd0af3662e None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:28.200525 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ddc6d2-5633-4b48-bc8c-4fdd0af3662e None admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:28.201303 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ddc6d2-5633-4b48-bc8c-4fdd0af3662e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:55:28.201648 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ddc6d2-5633-4b48-bc8c-4fdd0af3662e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:28.203090 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ddc6d2-5633-4b48-bc8c-4fdd0af3662e None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:28.215816 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 332/677] 10.222.0.46 () {56 vars in 1148 bytes} [Thu May 26 14:55:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:55:28.216703 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ddc6d2-5633-4b48-bc8c-4fdd0af3662e None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:28.287262 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee0fdec-d7ce-483d-85a1-40bff0c575ac None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:28.305420 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:28.316717 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:28.323634 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3429fdf-2a2f-47a6-ae02-1eacef2c5a66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:28.324435 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3429fdf-2a2f-47a6-ae02-1eacef2c5a66 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:28.324719 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3429fdf-2a2f-47a6-ae02-1eacef2c5a66 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:28.324996 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3429fdf-2a2f-47a6-ae02-1eacef2c5a66 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:28.332758 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ddc6d2-5633-4b48-bc8c-4fdd0af3662e None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:28.335710 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3429fdf-2a2f-47a6-ae02-1eacef2c5a66 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:28.337180 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3429fdf-2a2f-47a6-ae02-1eacef2c5a66 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:28.357563 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-b3429fdf-2a2f-47a6-ae02-1eacef2c5a66 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 14:55:28.357563 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:55:28.357563 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:55:28.357563 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:55:28.357563 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:55:28.357563 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:55:28.357563 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:55:28.357563 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:55:28.357563 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:55:28.357563 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:55:28.357563 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:55:28.357563 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:55:28.357563 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 14:55:28.357563 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 14:55:28.357563 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 14:55:28.358543 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 14:55:28.358543 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:55:28.358543 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:55:28.358543 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:55:28.358543 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:55:28.358543 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:55:28.358543 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:55:28.358543 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:55:28.358543 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 26 14:55:28.358543 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:55:28.358543 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 26 14:55:28.358543 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:55:28.358543 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:55:28.358543 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:55:28.358543 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 26 14:55:28.358543 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:55:28.358543 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 26 14:55:28.359420 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 14:55:28.359420 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 14:55:28.359420 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 14:55:28.359420 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 26 14:55:28.359420 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 346/678] 10.222.0.46 () {56 vars in 1108 bytes} [Thu May 26 14:55:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:55:28.359712 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3429fdf-2a2f-47a6-ae02-1eacef2c5a66 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:28.373396 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5413d608-773d-4a27-bf24-9e458f21c3ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:28.374553 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5413d608-773d-4a27-bf24-9e458f21c3ad None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:28.375011 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5413d608-773d-4a27-bf24-9e458f21c3ad None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:28.375292 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5413d608-773d-4a27-bf24-9e458f21c3ad None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:28.386294 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5413d608-773d-4a27-bf24-9e458f21c3ad None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:28.387768 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5413d608-773d-4a27-bf24-9e458f21c3ad None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:28.406253 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-5413d608-773d-4a27-bf24-9e458f21c3ad None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 14:55:28.406253 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:55:28.406253 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:55:28.406253 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:55:28.406253 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:55:28.406253 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:55:28.406253 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:55:28.406253 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:55:28.406253 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:55:28.406253 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:55:28.406253 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:55:28.406253 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:55:28.406253 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 14:55:28.406253 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 14:55:28.406253 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 14:55:28.407409 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 14:55:28.407409 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:55:28.407409 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:55:28.407409 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:55:28.407409 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:55:28.407409 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:55:28.407409 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:55:28.407409 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:55:28.407409 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 26 14:55:28.407409 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:55:28.407409 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 26 14:55:28.407409 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:55:28.407409 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:55:28.407409 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:55:28.407409 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 26 14:55:28.407409 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:55:28.407409 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 26 14:55:28.408493 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 14:55:28.408493 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 14:55:28.408493 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 14:55:28.408493 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 26 14:55:28.408493 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 333/679] 10.222.0.46 () {56 vars in 1108 bytes} [Thu May 26 14:55:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:55:28.408493 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5413d608-773d-4a27-bf24-9e458f21c3ad None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:28.483431 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3429fdf-2a2f-47a6-ae02-1eacef2c5a66 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:28.524528 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5413d608-773d-4a27-bf24-9e458f21c3ad None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:28.529135 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29f73cd-db49-4018-90ed-f8b9e7de6a89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:28.530554 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29f73cd-db49-4018-90ed-f8b9e7de6a89 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:28.531075 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29f73cd-db49-4018-90ed-f8b9e7de6a89 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:28.531560 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29f73cd-db49-4018-90ed-f8b9e7de6a89 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:28.538970 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29f73cd-db49-4018-90ed-f8b9e7de6a89 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:28.540455 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29f73cd-db49-4018-90ed-f8b9e7de6a89 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:28.556257 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-d29f73cd-db49-4018-90ed-f8b9e7de6a89 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 14:55:28.556257 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:55:28.556257 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:55:28.556257 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:55:28.556257 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:55:28.556257 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:55:28.556257 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:55:28.556257 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:55:28.556257 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:55:28.556257 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:55:28.556257 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:55:28.556257 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:55:28.556257 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 14:55:28.556257 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 14:55:28.556257 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 14:55:28.557840 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 14:55:28.557840 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:55:28.557840 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:55:28.557840 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:55:28.557840 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:55:28.557840 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:55:28.557840 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:55:28.557840 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:55:28.557840 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 26 14:55:28.557840 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:55:28.557840 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 26 14:55:28.557840 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:55:28.557840 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:55:28.557840 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:55:28.557840 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 26 14:55:28.557840 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:55:28.557840 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 26 14:55:28.558739 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 14:55:28.558739 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 14:55:28.558739 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 14:55:28.558739 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 14:55:28.558739 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 14:55:28.558739 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 14:55:28.558739 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 14:55:28.558739 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 26 14:55:28.559212 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:28.559296 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 347/680] 10.222.0.46 () {56 vars in 1111 bytes} [Thu May 26 14:55:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:55:28.559296 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29f73cd-db49-4018-90ed-f8b9e7de6a89 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:28.569195 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601c1353-5c43-4ca3-8de9-116205070807 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:28.569890 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601c1353-5c43-4ca3-8de9-116205070807 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:28.570175 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601c1353-5c43-4ca3-8de9-116205070807 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:28.570451 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601c1353-5c43-4ca3-8de9-116205070807 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:28.571288 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601c1353-5c43-4ca3-8de9-116205070807 None admin] RBAC: Adding query filter params (role.id=eb76213820fa4c4e9b0a5d5ac2a08b41, scope.project.id=a17fc8227d4c4da6b2847b350e912ee3, user.id=d6639353c70449d1b3d10cd6487c3b00) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:55:28.571598 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601c1353-5c43-4ca3-8de9-116205070807 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:28.573082 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601c1353-5c43-4ca3-8de9-116205070807 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:28.581013 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 334/681] 10.222.0.46 () {56 vars in 1374 bytes} [Thu May 26 14:55:28 2022] GET /identity/v3/role_assignments?user.id=d6639353c70449d1b3d10cd6487c3b00&scope.project.id=a17fc8227d4c4da6b2847b350e912ee3&role.id=eb76213820fa4c4e9b0a5d5ac2a08b41 => generated 608 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:55:28.582441 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601c1353-5c43-4ca3-8de9-116205070807 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:28.631197 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:28.704227 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29f73cd-db49-4018-90ed-f8b9e7de6a89 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:28.748203 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2f3c2e-36c2-41bb-9258-b700d0ab0935 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:28.753657 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2f3c2e-36c2-41bb-9258-b700d0ab0935 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:28.753657 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2f3c2e-36c2-41bb-9258-b700d0ab0935 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:28.753657 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2f3c2e-36c2-41bb-9258-b700d0ab0935 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:28.753657 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2f3c2e-36c2-41bb-9258-b700d0ab0935 None admin] RBAC: Adding query filter params (role.id=eb76213820fa4c4e9b0a5d5ac2a08b41, scope.project.id=a17fc8227d4c4da6b2847b350e912ee3, user.id=d5b5b35614a547d783e6e77ddb8c1c03) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:55:28.753657 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2f3c2e-36c2-41bb-9258-b700d0ab0935 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:28.753657 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2f3c2e-36c2-41bb-9258-b700d0ab0935 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:28.754610 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601c1353-5c43-4ca3-8de9-116205070807 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:28.777682 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 348/682] 10.222.0.46 () {56 vars in 1374 bytes} [Thu May 26 14:55:28 2022] GET /identity/v3/role_assignments?user.id=d5b5b35614a547d783e6e77ddb8c1c03&scope.project.id=a17fc8227d4c4da6b2847b350e912ee3&role.id=eb76213820fa4c4e9b0a5d5ac2a08b41 => generated 608 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:55:28.777682 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2f3c2e-36c2-41bb-9258-b700d0ab0935 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:28.818572 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b0363e-5bc9-4711-80f6-1f1a2efc60b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:28.819358 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b0363e-5bc9-4711-80f6-1f1a2efc60b9 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:28.819648 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b0363e-5bc9-4711-80f6-1f1a2efc60b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:28.819937 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b0363e-5bc9-4711-80f6-1f1a2efc60b9 None admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:28.820685 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b0363e-5bc9-4711-80f6-1f1a2efc60b9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:55:28.820996 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b0363e-5bc9-4711-80f6-1f1a2efc60b9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:28.823419 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b0363e-5bc9-4711-80f6-1f1a2efc60b9 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:28.841368 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 335/683] 10.222.0.46 () {56 vars in 1109 bytes} [Thu May 26 14:55:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:55:28.842335 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b0363e-5bc9-4711-80f6-1f1a2efc60b9 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:28.939874 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:28.956949 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2f3c2e-36c2-41bb-9258-b700d0ab0935 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:28.967657 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:29.001522 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b0363e-5bc9-4711-80f6-1f1a2efc60b9 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:29.003250 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbc2bde-31a8-4ade-a8cd-d08e46d327cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:29.004095 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbc2bde-31a8-4ade-a8cd-d08e46d327cd None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:29.004504 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbc2bde-31a8-4ade-a8cd-d08e46d327cd None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:29.005165 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbc2bde-31a8-4ade-a8cd-d08e46d327cd None admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:29.006347 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbc2bde-31a8-4ade-a8cd-d08e46d327cd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:55:29.006657 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbc2bde-31a8-4ade-a8cd-d08e46d327cd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:29.008053 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbc2bde-31a8-4ade-a8cd-d08e46d327cd None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:29.022576 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 349/684] 10.222.0.46 () {56 vars in 1109 bytes} [Thu May 26 14:55:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:55:29.023457 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbc2bde-31a8-4ade-a8cd-d08e46d327cd None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:29.046182 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54baf1db-1427-4dc5-8abb-0c84b188b607 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:29.046962 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54baf1db-1427-4dc5-8abb-0c84b188b607 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:29.047376 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54baf1db-1427-4dc5-8abb-0c84b188b607 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:29.047690 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54baf1db-1427-4dc5-8abb-0c84b188b607 None admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:29.048452 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54baf1db-1427-4dc5-8abb-0c84b188b607 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:55:29.048761 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54baf1db-1427-4dc5-8abb-0c84b188b607 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:29.050265 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54baf1db-1427-4dc5-8abb-0c84b188b607 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:29.067486 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 336/685] 10.222.0.46 () {56 vars in 1148 bytes} [Thu May 26 14:55:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:55:29.067696 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54baf1db-1427-4dc5-8abb-0c84b188b607 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:29.175205 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbc2bde-31a8-4ade-a8cd-d08e46d327cd None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:29.217174 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54baf1db-1427-4dc5-8abb-0c84b188b607 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:29.220172 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe827f4-8c59-4f3c-9e36-2e5f834dc914 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:29.221004 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe827f4-8c59-4f3c-9e36-2e5f834dc914 None admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:29.221364 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe827f4-8c59-4f3c-9e36-2e5f834dc914 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:29.221905 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe827f4-8c59-4f3c-9e36-2e5f834dc914 None admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:29.222820 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe827f4-8c59-4f3c-9e36-2e5f834dc914 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:29.224416 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe827f4-8c59-4f3c-9e36-2e5f834dc914 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:29.242221 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:29.264934 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 350/686] 10.222.0.46 () {60 vars in 1132 bytes} [Thu May 26 14:55:29 2022] POST /identity/v3/users => generated 254 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 14:55:29.266779 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe827f4-8c59-4f3c-9e36-2e5f834dc914 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:29.282959 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38c284e-4ba7-424f-84c3-8ba014515507 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:29.284142 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38c284e-4ba7-424f-84c3-8ba014515507 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:29.284681 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38c284e-4ba7-424f-84c3-8ba014515507 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:29.285371 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38c284e-4ba7-424f-84c3-8ba014515507 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:29.297745 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38c284e-4ba7-424f-84c3-8ba014515507 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:29.299933 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38c284e-4ba7-424f-84c3-8ba014515507 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:29.317566 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-c38c284e-4ba7-424f-84c3-8ba014515507 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 26 14:55:29.317566 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:55:29.317566 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:55:29.317566 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:55:29.317566 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:55:29.317566 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:55:29.317566 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:55:29.317566 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:55:29.317566 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:55:29.317566 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:55:29.317566 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:55:29.317566 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:55:29.317566 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 14:55:29.317566 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 14:55:29.317566 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 14:55:29.317566 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 14:55:29.318792 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:55:29.318792 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:55:29.318792 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 14:55:29.318792 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 14:55:29.318792 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 14:55:29.318792 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 14:55:29.318792 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:55:29.318792 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:55:29.318792 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:55:29.318792 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:55:29.318792 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:55:29.318792 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 26 14:55:29.318792 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:55:29.318792 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 26 14:55:29.318792 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:55:29.318792 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:55:29.318792 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:55:29.318792 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 26 14:55:29.319920 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:55:29.319920 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 26 14:55:29.319920 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 14:55:29.319920 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 14:55:29.319920 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 14:55:29.319920 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 14:55:29.319920 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 14:55:29.319920 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 14:55:29.319920 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 26 14:55:29.319920 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 26 14:55:29.323541 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 337/687] 10.222.0.46 () {56 vars in 1096 bytes} [Thu May 26 14:55:29 2022] GET /identity/v3/users/swift => generated 83 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:55:29.331538 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38c284e-4ba7-424f-84c3-8ba014515507 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:29.338118 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:29.506448 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe827f4-8c59-4f3c-9e36-2e5f834dc914 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:29.561802 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0393921a-828e-43b1-8e6a-0fc813be7f76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:29.564097 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0393921a-828e-43b1-8e6a-0fc813be7f76 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:29.566506 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38c284e-4ba7-424f-84c3-8ba014515507 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:29.567128 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0393921a-828e-43b1-8e6a-0fc813be7f76 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:29.567557 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0393921a-828e-43b1-8e6a-0fc813be7f76 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:29.568646 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0393921a-828e-43b1-8e6a-0fc813be7f76 None admin] RBAC: Adding query filter params (role.id=eb76213820fa4c4e9b0a5d5ac2a08b41, scope.project.id=a17fc8227d4c4da6b2847b350e912ee3, user.id=0156a2f6f3794f479e76b85bdbbe5c26) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:55:29.568963 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0393921a-828e-43b1-8e6a-0fc813be7f76 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:29.570474 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0393921a-828e-43b1-8e6a-0fc813be7f76 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:29.580140 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 351/688] 10.222.0.46 () {56 vars in 1374 bytes} [Thu May 26 14:55:29 2022] GET /identity/v3/role_assignments?user.id=0156a2f6f3794f479e76b85bdbbe5c26&scope.project.id=a17fc8227d4c4da6b2847b350e912ee3&role.id=eb76213820fa4c4e9b0a5d5ac2a08b41 => generated 608 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:55:29.640312 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771d940b-3dfe-47f4-90a5-88053e981a9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:29.641274 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771d940b-3dfe-47f4-90a5-88053e981a9d None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:29.641775 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771d940b-3dfe-47f4-90a5-88053e981a9d None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:29.642207 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771d940b-3dfe-47f4-90a5-88053e981a9d None admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:29.643963 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771d940b-3dfe-47f4-90a5-88053e981a9d None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:55:29.644507 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771d940b-3dfe-47f4-90a5-88053e981a9d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:29.646206 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771d940b-3dfe-47f4-90a5-88053e981a9d None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:29.669907 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 338/689] 10.222.0.46 () {56 vars in 1135 bytes} [Thu May 26 14:55:29 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 339 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:55:29.675450 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0393921a-828e-43b1-8e6a-0fc813be7f76 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:29.794605 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:29.815295 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0393921a-828e-43b1-8e6a-0fc813be7f76 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:29.856413 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40112a42-fe92-4acc-bb4f-41b77db13867 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:29.857427 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40112a42-fe92-4acc-bb4f-41b77db13867 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:29.857967 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40112a42-fe92-4acc-bb4f-41b77db13867 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:29.858447 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40112a42-fe92-4acc-bb4f-41b77db13867 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:29.943154 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40112a42-fe92-4acc-bb4f-41b77db13867 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:29.945010 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40112a42-fe92-4acc-bb4f-41b77db13867 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:30.044083 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:30.063518 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 352/690] 10.222.0.46 () {58 vars in 1282 bytes} [Thu May 26 14:55:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 14:55:30.071342 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771d940b-3dfe-47f4-90a5-88053e981a9d None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:30.115340 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:30.245058 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771d940b-3dfe-47f4-90a5-88053e981a9d None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:30.308180 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e23e5ce-0bcc-4332-80eb-6d9b6c18dffa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:30.308993 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e23e5ce-0bcc-4332-80eb-6d9b6c18dffa None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:30.309277 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e23e5ce-0bcc-4332-80eb-6d9b6c18dffa None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:30.309576 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e23e5ce-0bcc-4332-80eb-6d9b6c18dffa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:30.327055 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e23e5ce-0bcc-4332-80eb-6d9b6c18dffa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:30.328458 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e23e5ce-0bcc-4332-80eb-6d9b6c18dffa None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:30.358488 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-8e23e5ce-0bcc-4332-80eb-6d9b6c18dffa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 14:55:30.358488 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:55:30.358488 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:55:30.358488 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:55:30.358488 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:55:30.358488 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:55:30.358488 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:55:30.358488 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:55:30.358488 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:55:30.358488 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:55:30.358488 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:55:30.358488 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:55:30.358488 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 14:55:30.358488 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 14:55:30.358488 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 14:55:30.359467 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 14:55:30.359467 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:55:30.359467 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:55:30.359467 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:55:30.359467 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:55:30.359467 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:55:30.359467 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:55:30.359467 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:55:30.359467 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 26 14:55:30.359467 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:55:30.359467 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 26 14:55:30.359467 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:55:30.359467 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:55:30.359467 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:55:30.359467 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 26 14:55:30.359467 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:55:30.359467 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 26 14:55:30.360385 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 14:55:30.360385 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 14:55:30.360385 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 14:55:30.360385 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 26 14:55:30.364268 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 339/691] 10.222.0.46 () {56 vars in 1108 bytes} [Thu May 26 14:55:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:55:30.367450 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40112a42-fe92-4acc-bb4f-41b77db13867 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:30.525476 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40112a42-fe92-4acc-bb4f-41b77db13867 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:30.580892 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7efb54-6ea1-4920-9462-98e64bcd5fb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:30.586055 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7efb54-6ea1-4920-9462-98e64bcd5fb0 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:30.586610 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7efb54-6ea1-4920-9462-98e64bcd5fb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:30.587092 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7efb54-6ea1-4920-9462-98e64bcd5fb0 None admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:30.588043 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7efb54-6ea1-4920-9462-98e64bcd5fb0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:55:30.588609 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7efb54-6ea1-4920-9462-98e64bcd5fb0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:30.590261 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7efb54-6ea1-4920-9462-98e64bcd5fb0 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:30.603597 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 353/692] 10.222.0.46 () {56 vars in 1109 bytes} [Thu May 26 14:55:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:55:30.613606 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e23e5ce-0bcc-4332-80eb-6d9b6c18dffa None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:30.738790 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:30.758792 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e23e5ce-0bcc-4332-80eb-6d9b6c18dffa None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:30.802228 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5865c747-2127-47f0-bbb2-60bfc4985fdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:30.803059 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5865c747-2127-47f0-bbb2-60bfc4985fdd None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:30.803344 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5865c747-2127-47f0-bbb2-60bfc4985fdd None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:30.803620 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5865c747-2127-47f0-bbb2-60bfc4985fdd None admin] PATH_INFO: `/v3/projects/service` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:30.811700 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5865c747-2127-47f0-bbb2-60bfc4985fdd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:30.813306 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5865c747-2127-47f0-bbb2-60bfc4985fdd None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:30.829175 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-5865c747-2127-47f0-bbb2-60bfc4985fdd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 14:55:30.829175 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:55:30.829175 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:55:30.829175 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:55:30.829175 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:55:30.829175 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:55:30.829175 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:55:30.829175 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:55:30.829175 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:55:30.829175 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:55:30.829175 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:55:30.829175 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:55:30.829175 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 14:55:30.829175 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 14:55:30.829175 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 14:55:30.830174 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 14:55:30.830174 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:55:30.830174 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:55:30.830174 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:55:30.830174 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:55:30.830174 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:55:30.830174 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:55:30.830174 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:55:30.830174 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 26 14:55:30.830174 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:55:30.830174 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 26 14:55:30.830174 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:55:30.830174 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:55:30.830174 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:55:30.830174 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 26 14:55:30.830174 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:55:30.830174 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 26 14:55:30.831104 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 14:55:30.831104 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 14:55:30.831104 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 14:55:30.831104 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 14:55:30.831104 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 14:55:30.831104 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 14:55:30.831104 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 14:55:30.831104 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 26 14:55:30.831104 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 340/693] 10.222.0.46 () {56 vars in 1111 bytes} [Thu May 26 14:55:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:55:30.839398 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7efb54-6ea1-4920-9462-98e64bcd5fb0 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:30.937644 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:31.016493 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7efb54-6ea1-4920-9462-98e64bcd5fb0 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:31.068130 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d5a2f8-840a-4ad2-9a03-d465f5ac4e94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:31.068823 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d5a2f8-840a-4ad2-9a03-d465f5ac4e94 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:31.069113 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d5a2f8-840a-4ad2-9a03-d465f5ac4e94 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:31.069397 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d5a2f8-840a-4ad2-9a03-d465f5ac4e94 None admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:31.070230 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d5a2f8-840a-4ad2-9a03-d465f5ac4e94 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:55:31.070542 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d5a2f8-840a-4ad2-9a03-d465f5ac4e94 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:31.071988 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d5a2f8-840a-4ad2-9a03-d465f5ac4e94 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:31.096166 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 354/694] 10.222.0.46 () {56 vars in 1148 bytes} [Thu May 26 14:55:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:55:31.102343 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5865c747-2127-47f0-bbb2-60bfc4985fdd None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:31.277859 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5865c747-2127-47f0-bbb2-60bfc4985fdd None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:31.335006 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79614957-49a2-4a8f-8577-1847d0be3e08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:31.336020 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79614957-49a2-4a8f-8577-1847d0be3e08 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:31.338187 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79614957-49a2-4a8f-8577-1847d0be3e08 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:31.339259 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79614957-49a2-4a8f-8577-1847d0be3e08 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:31.340711 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79614957-49a2-4a8f-8577-1847d0be3e08 None admin] RBAC: Adding query filter params (role.id=eb76213820fa4c4e9b0a5d5ac2a08b41, scope.project.id=a17fc8227d4c4da6b2847b350e912ee3, user.id=a4bea272624b45d8ac2ad27bd07df885) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:55:31.341046 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79614957-49a2-4a8f-8577-1847d0be3e08 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:31.343221 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79614957-49a2-4a8f-8577-1847d0be3e08 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:31.355602 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 341/695] 10.222.0.46 () {56 vars in 1374 bytes} [Thu May 26 14:55:31 2022] GET /identity/v3/role_assignments?user.id=a4bea272624b45d8ac2ad27bd07df885&scope.project.id=a17fc8227d4c4da6b2847b350e912ee3&role.id=eb76213820fa4c4e9b0a5d5ac2a08b41 => generated 608 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:55:31.440158 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627089ed-9156-4a8c-9edc-f054f23551cd None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:31.440520 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627089ed-9156-4a8c-9edc-f054f23551cd None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:31.440764 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627089ed-9156-4a8c-9edc-f054f23551cd None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:31.442562 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 355/696] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 14:55:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 14:55:31.448778 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e963ba1-d2dc-4129-8ac9-4870a1ff33bc None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:31.449081 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e963ba1-d2dc-4129-8ac9-4870a1ff33bc None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:31.449343 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e963ba1-d2dc-4129-8ac9-4870a1ff33bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:31.474683 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-9e963ba1-d2dc-4129-8ac9-4870a1ff33bc None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:55:31.475937 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-9e963ba1-d2dc-4129-8ac9-4870a1ff33bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:55:31.478259 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:31.577781 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 342/697] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 14:55:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 14:55:31.627554 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627089ed-9156-4a8c-9edc-f054f23551cd None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:31.792453 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627089ed-9156-4a8c-9edc-f054f23551cd None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:31.802823 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:31.845074 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7604947e-37c4-4a99-a55f-b8c2931e886e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:31.845923 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7604947e-37c4-4a99-a55f-b8c2931e886e None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:31.846213 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7604947e-37c4-4a99-a55f-b8c2931e886e None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:31.846495 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7604947e-37c4-4a99-a55f-b8c2931e886e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:31.856344 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643f7544-1520-411f-879e-94db6c31ffc8 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:31.856642 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643f7544-1520-411f-879e-94db6c31ffc8 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:31.857056 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643f7544-1520-411f-879e-94db6c31ffc8 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:31.858976 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 343/698] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 14:55:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 14:55:31.863645 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7604947e-37c4-4a99-a55f-b8c2931e886e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 14:55:31.864664 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687901a5-4525-4215-ac88-4f15d77f4944 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:31.864973 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687901a5-4525-4215-ac88-4f15d77f4944 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:31.865238 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687901a5-4525-4215-ac88-4f15d77f4944 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:31.872890 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-7604947e-37c4-4a99-a55f-b8c2931e886e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 14:55:31.872890 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:55:31.872890 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:55:31.872890 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:55:31.872890 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:55:31.872890 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:55:31.872890 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:55:31.872890 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:55:31.872890 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:55:31.872890 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:55:31.872890 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:55:31.872890 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:55:31.872890 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 14:55:31.872890 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 14:55:31.872890 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 14:55:31.872890 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 14:55:31.873926 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 14:55:31.873926 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 14:55:31.873926 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 14:55:31.873926 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 14:55:31.873926 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:55:31.873926 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:55:31.873926 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:55:31.873926 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:55:31.873926 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:55:31.873926 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:55:31.873926 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:55:31.873926 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 26 14:55:31.873926 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:55:31.873926 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 26 14:55:31.873926 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:55:31.873926 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:55:31.874838 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:55:31.874838 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 26 14:55:31.874838 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:55:31.874838 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 26 14:55:31.874838 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 14:55:31.874838 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 14:55:31.874838 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 14:55:31.874838 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 14:55:31.874838 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 14:55:31.874838 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 14:55:31.874838 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 14:55:31.874838 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 26 14:55:31.879034 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 356/699] 10.222.0.46 () {56 vars in 1096 bytes} [Thu May 26 14:55:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:55:31.880714 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7604947e-37c4-4a99-a55f-b8c2931e886e None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:31.891526 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-687901a5-4525-4215-ac88-4f15d77f4944 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:55:31.893235 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-687901a5-4525-4215-ac88-4f15d77f4944 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:55:32.002804 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 344/700] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 14:55:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 26 14:55:32.053048 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7604947e-37c4-4a99-a55f-b8c2931e886e None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:32.064685 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687901a5-4525-4215-ac88-4f15d77f4944 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:32.097005 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b24eb34-7982-4e77-b4c6-ade92f6c4846 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:32.097834 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b24eb34-7982-4e77-b4c6-ade92f6c4846 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:32.098222 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b24eb34-7982-4e77-b4c6-ade92f6c4846 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:32.098579 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b24eb34-7982-4e77-b4c6-ade92f6c4846 None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:32.100044 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b24eb34-7982-4e77-b4c6-ade92f6c4846 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:55:32.100427 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b24eb34-7982-4e77-b4c6-ade92f6c4846 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:32.104564 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b24eb34-7982-4e77-b4c6-ade92f6c4846 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:32.115090 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 357/701] 10.222.0.46 () {56 vars in 1099 bytes} [Thu May 26 14:55:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:55:32.123805 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b24eb34-7982-4e77-b4c6-ade92f6c4846 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:32.186850 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:32.193836 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:32.234095 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687901a5-4525-4215-ac88-4f15d77f4944 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:32.293435 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9347219-8aa3-4b20-a1e3-3b5408b874dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:32.295850 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9347219-8aa3-4b20-a1e3-3b5408b874dd None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:32.296158 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9347219-8aa3-4b20-a1e3-3b5408b874dd None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:32.296598 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9347219-8aa3-4b20-a1e3-3b5408b874dd None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:32.297288 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9347219-8aa3-4b20-a1e3-3b5408b874dd None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:32.298740 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9347219-8aa3-4b20-a1e3-3b5408b874dd None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:32.318096 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-a9347219-8aa3-4b20-a1e3-3b5408b874dd None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 26 14:55:32.318096 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:55:32.318096 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 14:55:32.318096 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 14:55:32.318096 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 14:55:32.318096 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 14:55:32.318096 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 14:55:32.318096 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 14:55:32.318096 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 26 14:55:32.318096 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 26 14:55:32.318096 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 14:55:32.318096 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 26 14:55:32.318096 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:55:32.318096 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:55:32.318096 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:55:32.318096 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:55:32.320239 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:55:32.320239 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:55:32.320239 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:55:32.320239 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:55:32.320239 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:55:32.320239 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:55:32.320239 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:55:32.320239 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 14:55:32.320239 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 14:55:32.320239 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 14:55:32.320239 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 14:55:32.320239 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:55:32.320239 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:55:32.320239 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:55:32.320239 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:55:32.320239 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:55:32.321984 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:55:32.321984 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:55:32.321984 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 26 14:55:32.321984 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:55:32.321984 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 26 14:55:32.321984 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:55:32.321984 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:55:32.321984 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:55:32.321984 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 26 14:55:32.321984 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:55:32.321984 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 26 14:55:32.321984 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 14:55:32.321984 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 14:55:32.321984 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 26 14:55:32.321984 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 26 14:55:32.321984 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 345/702] 10.222.0.46 () {56 vars in 1129 bytes} [Thu May 26 14:55:32 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:55:32.330102 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749abd4b-5e7f-4d7e-b9f9-008bf74c3151 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:32.330497 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749abd4b-5e7f-4d7e-b9f9-008bf74c3151 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:32.330779 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749abd4b-5e7f-4d7e-b9f9-008bf74c3151 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:32.332029 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 346/703] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 14:55:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 14:55:32.342186 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749abd4b-5e7f-4d7e-b9f9-008bf74c3151 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:32.372526 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b24eb34-7982-4e77-b4c6-ade92f6c4846 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:32.428362 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9d5a4d-13db-4b62-b59a-982001d9cc48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:32.429174 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9d5a4d-13db-4b62-b59a-982001d9cc48 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:32.429481 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9d5a4d-13db-4b62-b59a-982001d9cc48 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:32.429768 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9d5a4d-13db-4b62-b59a-982001d9cc48 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:32.444321 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9d5a4d-13db-4b62-b59a-982001d9cc48 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:32.445911 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9d5a4d-13db-4b62-b59a-982001d9cc48 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:32.472860 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-3f9d5a4d-13db-4b62-b59a-982001d9cc48 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 14:55:32.472860 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:55:32.472860 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:55:32.472860 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:55:32.472860 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:55:32.472860 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:55:32.472860 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:55:32.472860 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:55:32.472860 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:55:32.472860 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:55:32.472860 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:55:32.472860 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:55:32.472860 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 14:55:32.472860 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 14:55:32.472860 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 14:55:32.475127 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 14:55:32.475127 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:55:32.475127 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:55:32.475127 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:55:32.475127 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:55:32.475127 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:55:32.475127 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:55:32.475127 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:55:32.475127 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 26 14:55:32.475127 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:55:32.475127 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 26 14:55:32.475127 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:55:32.475127 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:55:32.475127 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:55:32.475127 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 26 14:55:32.475127 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:55:32.475127 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 26 14:55:32.477251 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 14:55:32.477251 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 14:55:32.477251 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 14:55:32.477251 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 26 14:55:32.477251 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 358/704] 10.222.0.46 () {56 vars in 1108 bytes} [Thu May 26 14:55:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 351 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:55:32.477251 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35185db-567d-4ba6-b15f-7c0c6458f0fc None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:32.477251 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35185db-567d-4ba6-b15f-7c0c6458f0fc None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:32.477251 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35185db-567d-4ba6-b15f-7c0c6458f0fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:32.494894 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749abd4b-5e7f-4d7e-b9f9-008bf74c3151 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:32.505720 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-f35185db-567d-4ba6-b15f-7c0c6458f0fc None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:55:32.508256 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-f35185db-567d-4ba6-b15f-7c0c6458f0fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:55:32.535820 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:32.543193 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43fb73e-62c3-4872-8bec-038146e70679 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:32.544247 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43fb73e-62c3-4872-8bec-038146e70679 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:32.544909 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43fb73e-62c3-4872-8bec-038146e70679 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:32.545393 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43fb73e-62c3-4872-8bec-038146e70679 None admin] PATH_INFO: `/v3/services` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:32.546386 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43fb73e-62c3-4872-8bec-038146e70679 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:55:32.546899 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43fb73e-62c3-4872-8bec-038146e70679 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:32.548702 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d43fb73e-62c3-4872-8bec-038146e70679 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:32.555801 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 347/705] 10.222.0.46 () {56 vars in 1124 bytes} [Thu May 26 14:55:32 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:55:32.557045 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43fb73e-62c3-4872-8bec-038146e70679 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:32.660188 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 359/706] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 14:55:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 14:55:32.661850 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35185db-567d-4ba6-b15f-7c0c6458f0fc None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:32.686094 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:32.688269 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d43fb73e-62c3-4872-8bec-038146e70679 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:32.734284 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e3e923-4fb7-464b-8ad8-6c424948b67f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:32.735409 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e3e923-4fb7-464b-8ad8-6c424948b67f None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:32.736014 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e3e923-4fb7-464b-8ad8-6c424948b67f None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:32.736536 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e3e923-4fb7-464b-8ad8-6c424948b67f None admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:32.737795 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e3e923-4fb7-464b-8ad8-6c424948b67f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:55:32.738374 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e3e923-4fb7-464b-8ad8-6c424948b67f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:32.740078 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e3e923-4fb7-464b-8ad8-6c424948b67f None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:32.867910 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35185db-567d-4ba6-b15f-7c0c6458f0fc None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:32.887846 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 348/707] 10.222.0.46 () {56 vars in 1109 bytes} [Thu May 26 14:55:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 327 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:55:32.889181 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e3e923-4fb7-464b-8ad8-6c424948b67f None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:32.930289 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f2028b-0892-43e8-8341-50e6e91853ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:32.931443 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f2028b-0892-43e8-8341-50e6e91853ab None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:32.932056 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f2028b-0892-43e8-8341-50e6e91853ab None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:32.932488 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f2028b-0892-43e8-8341-50e6e91853ab None admin] PATH_INFO: `/v3/services` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:32.933817 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f2028b-0892-43e8-8341-50e6e91853ab None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:55:32.934842 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f2028b-0892-43e8-8341-50e6e91853ab None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:32.936917 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f2028b-0892-43e8-8341-50e6e91853ab None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:32.953525 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 360/708] 10.222.0.46 () {56 vars in 1124 bytes} [Thu May 26 14:55:32 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:55:32.953525 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f2028b-0892-43e8-8341-50e6e91853ab None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:32.964180 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:33.056513 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e3e923-4fb7-464b-8ad8-6c424948b67f None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:33.094494 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:33.119063 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fef53b-8e73-4cdd-9c35-90b5e448161e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:33.119953 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fef53b-8e73-4cdd-9c35-90b5e448161e None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:33.120307 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fef53b-8e73-4cdd-9c35-90b5e448161e None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:33.120590 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fef53b-8e73-4cdd-9c35-90b5e448161e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:33.121955 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f2028b-0892-43e8-8341-50e6e91853ab None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:33.141625 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fef53b-8e73-4cdd-9c35-90b5e448161e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 14:55:33.151395 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-17fef53b-8e73-4cdd-9c35-90b5e448161e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 26 14:55:33.151395 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:55:33.151395 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:55:33.151395 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:55:33.151395 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:55:33.151395 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:55:33.151395 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:55:33.151395 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:55:33.151395 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:55:33.151395 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:55:33.151395 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:55:33.151395 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:55:33.151395 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 14:55:33.151395 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 14:55:33.151395 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 14:55:33.151395 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 14:55:33.152488 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 14:55:33.152488 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 14:55:33.152488 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 14:55:33.152488 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 14:55:33.152488 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:55:33.152488 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:55:33.152488 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:55:33.152488 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:55:33.152488 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:55:33.152488 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:55:33.152488 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:55:33.152488 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 26 14:55:33.152488 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:55:33.152488 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 26 14:55:33.152488 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:55:33.152488 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:55:33.153409 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:55:33.153409 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 26 14:55:33.153409 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:55:33.153409 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 26 14:55:33.153409 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 14:55:33.153409 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 14:55:33.153409 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 14:55:33.153409 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 14:55:33.153409 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 14:55:33.153409 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 14:55:33.153409 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 26 14:55:33.153409 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 26 14:55:33.156601 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 349/709] 10.222.0.46 () {56 vars in 1102 bytes} [Thu May 26 14:55:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:55:33.158725 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8fe51f-2ba2-475d-b14e-ae0ae6a667f9 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:33.159556 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8fe51f-2ba2-475d-b14e-ae0ae6a667f9 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:33.160340 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8fe51f-2ba2-475d-b14e-ae0ae6a667f9 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:33.162652 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 350/710] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 14:55:33 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 14:55:33.173240 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b6243d-7771-4e24-ab1b-c80b72a1cd1a None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:33.173696 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b6243d-7771-4e24-ab1b-c80b72a1cd1a None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:33.173869 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b6243d-7771-4e24-ab1b-c80b72a1cd1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:33.193504 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e93eea-12a0-4a5b-a0ae-5a091f010914 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:33.193504 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e93eea-12a0-4a5b-a0ae-5a091f010914 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:33.193939 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e93eea-12a0-4a5b-a0ae-5a091f010914 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:33.194558 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e93eea-12a0-4a5b-a0ae-5a091f010914 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:33.198667 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-70b6243d-7771-4e24-ab1b-c80b72a1cd1a None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:55:33.199903 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-70b6243d-7771-4e24-ab1b-c80b72a1cd1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:55:33.210376 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e93eea-12a0-4a5b-a0ae-5a091f010914 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:33.212378 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e93eea-12a0-4a5b-a0ae-5a091f010914 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:33.232846 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-15e93eea-12a0-4a5b-a0ae-5a091f010914 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 26 14:55:33.232846 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:55:33.232846 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:55:33.232846 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:55:33.232846 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:55:33.232846 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:55:33.232846 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:55:33.232846 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:55:33.232846 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:55:33.232846 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:55:33.232846 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:55:33.232846 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:55:33.232846 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 14:55:33.232846 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 14:55:33.232846 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 14:55:33.232846 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 14:55:33.234001 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:55:33.234001 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:55:33.234001 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 14:55:33.234001 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 14:55:33.234001 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 14:55:33.234001 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 14:55:33.234001 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:55:33.234001 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:55:33.234001 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:55:33.234001 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:55:33.234001 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:55:33.234001 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 26 14:55:33.234001 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:55:33.234001 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 26 14:55:33.234001 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:55:33.234001 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:55:33.234001 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:55:33.234001 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 26 14:55:33.234954 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:55:33.234954 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 26 14:55:33.234954 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 14:55:33.234954 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 14:55:33.234954 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 14:55:33.234954 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 14:55:33.234954 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 14:55:33.234954 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 14:55:33.234954 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 26 14:55:33.234954 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 26 14:55:33.234954 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 361/711] 10.222.0.46 () {56 vars in 1093 bytes} [Thu May 26 14:55:32 2022] GET /identity/v3/users/nova => generated 82 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:55:33.236284 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e93eea-12a0-4a5b-a0ae-5a091f010914 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:33.310463 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 351/712] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 14:55:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 14:55:33.311456 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b6243d-7771-4e24-ab1b-c80b72a1cd1a None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:33.394056 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:33.402874 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e93eea-12a0-4a5b-a0ae-5a091f010914 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:33.454401 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d2326c-9256-40de-8201-6010103a1bf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:33.455232 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d2326c-9256-40de-8201-6010103a1bf5 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:33.455586 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d2326c-9256-40de-8201-6010103a1bf5 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:33.455866 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d2326c-9256-40de-8201-6010103a1bf5 None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:33.457136 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d2326c-9256-40de-8201-6010103a1bf5 None admin] RBAC: Adding query filter params (name=service) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:55:33.457545 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d2326c-9256-40de-8201-6010103a1bf5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:33.459204 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d2326c-9256-40de-8201-6010103a1bf5 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:33.461154 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:33.476613 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 362/713] 10.222.0.46 () {56 vars in 1103 bytes} [Thu May 26 14:55:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:55:33.478186 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d2326c-9256-40de-8201-6010103a1bf5 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:33.484669 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b6243d-7771-4e24-ab1b-c80b72a1cd1a None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:33.540564 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7eee3e-ce47-4056-bf62-2fecd70c712d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:33.541672 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7eee3e-ce47-4056-bf62-2fecd70c712d None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:33.542165 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7eee3e-ce47-4056-bf62-2fecd70c712d None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:33.543304 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7eee3e-ce47-4056-bf62-2fecd70c712d None admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:33.544388 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7eee3e-ce47-4056-bf62-2fecd70c712d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:55:33.544919 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7eee3e-ce47-4056-bf62-2fecd70c712d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:33.548093 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7eee3e-ce47-4056-bf62-2fecd70c712d None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:33.577193 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 352/714] 10.222.0.46 () {56 vars in 1133 bytes} [Thu May 26 14:55:33 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:55:33.579809 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7eee3e-ce47-4056-bf62-2fecd70c712d None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:33.647671 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d2326c-9256-40de-8201-6010103a1bf5 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:33.701752 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e4d0d9-a11a-4084-8b3d-df1ec581e842 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:33.702844 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e4d0d9-a11a-4084-8b3d-df1ec581e842 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:33.703196 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e4d0d9-a11a-4084-8b3d-df1ec581e842 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:33.703548 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e4d0d9-a11a-4084-8b3d-df1ec581e842 None admin] PATH_INFO: `/v3/services/network` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:33.704397 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e4d0d9-a11a-4084-8b3d-df1ec581e842 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:33.706294 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e4d0d9-a11a-4084-8b3d-df1ec581e842 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:33.730541 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-f7e4d0d9-a11a-4084-8b3d-df1ec581e842 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 26 14:55:33.730541 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:55:33.730541 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 14:55:33.730541 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 14:55:33.730541 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 14:55:33.730541 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 14:55:33.730541 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 14:55:33.730541 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 14:55:33.730541 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 26 14:55:33.730541 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 26 14:55:33.730541 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 14:55:33.730541 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 26 14:55:33.730541 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:55:33.730541 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:55:33.730541 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:55:33.730541 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:55:33.730541 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:55:33.731829 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:55:33.731829 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:55:33.731829 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:55:33.731829 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:55:33.731829 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:55:33.731829 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:55:33.731829 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 14:55:33.731829 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 14:55:33.731829 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 14:55:33.731829 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 14:55:33.731829 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:55:33.731829 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:55:33.731829 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:55:33.731829 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:55:33.731829 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:55:33.731829 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:55:33.732783 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:55:33.732783 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 26 14:55:33.732783 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:55:33.732783 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 26 14:55:33.732783 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:55:33.732783 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:55:33.732783 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:55:33.732783 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 26 14:55:33.732783 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:55:33.732783 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 26 14:55:33.732783 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 14:55:33.732783 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 14:55:33.732783 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 26 14:55:33.732783 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 26 14:55:33.737961 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 363/715] 10.222.0.46 () {56 vars in 1111 bytes} [Thu May 26 14:55:33 2022] GET /identity/v3/services/network => generated 88 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:55:33.739110 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e4d0d9-a11a-4084-8b3d-df1ec581e842 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:33.759091 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7eee3e-ce47-4056-bf62-2fecd70c712d None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:33.769824 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:33.806762 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53362762-ba3c-4f11-a17f-ed879690b7b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:33.807621 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53362762-ba3c-4f11-a17f-ed879690b7b7 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:33.807904 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53362762-ba3c-4f11-a17f-ed879690b7b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:33.808188 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53362762-ba3c-4f11-a17f-ed879690b7b7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:33.821858 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53362762-ba3c-4f11-a17f-ed879690b7b7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:33.824585 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53362762-ba3c-4f11-a17f-ed879690b7b7 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:33.840822 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:33.852529 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-53362762-ba3c-4f11-a17f-ed879690b7b7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 14:55:33.852529 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:55:33.852529 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:55:33.852529 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:55:33.852529 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:55:33.852529 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:55:33.852529 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:55:33.852529 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:55:33.852529 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:55:33.852529 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:55:33.852529 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:55:33.852529 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:55:33.852529 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 14:55:33.852529 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 14:55:33.852529 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 14:55:33.853792 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 14:55:33.853792 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:55:33.853792 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:55:33.853792 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:55:33.853792 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:55:33.853792 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:55:33.853792 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:55:33.853792 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:55:33.853792 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 26 14:55:33.853792 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:55:33.853792 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 26 14:55:33.853792 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:55:33.853792 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:55:33.853792 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:55:33.853792 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 26 14:55:33.853792 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:55:33.853792 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 26 14:55:33.854718 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 14:55:33.854718 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 14:55:33.854718 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 14:55:33.854718 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 26 14:55:33.857823 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 353/716] 10.222.0.46 () {56 vars in 1108 bytes} [Thu May 26 14:55:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:55:33.858861 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53362762-ba3c-4f11-a17f-ed879690b7b7 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:33.896157 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e4d0d9-a11a-4084-8b3d-df1ec581e842 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:33.943615 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60b41aa-b1e1-4572-a063-547818b265bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:33.945079 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60b41aa-b1e1-4572-a063-547818b265bd None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:33.945790 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60b41aa-b1e1-4572-a063-547818b265bd None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:33.946381 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60b41aa-b1e1-4572-a063-547818b265bd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:33.990612 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53362762-ba3c-4f11-a17f-ed879690b7b7 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:34.033713 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438ed032-1e82-4e81-b43d-8a2ec7058fee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:34.034395 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438ed032-1e82-4e81-b43d-8a2ec7058fee None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:34.034680 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438ed032-1e82-4e81-b43d-8a2ec7058fee None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:34.034957 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-438ed032-1e82-4e81-b43d-8a2ec7058fee None admin] PATH_INFO: `/v3/services` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:34.035677 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438ed032-1e82-4e81-b43d-8a2ec7058fee None admin] RBAC: Adding query filter params (name=network) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:55:34.035986 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438ed032-1e82-4e81-b43d-8a2ec7058fee None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:34.037623 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-438ed032-1e82-4e81-b43d-8a2ec7058fee None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:34.040911 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60b41aa-b1e1-4572-a063-547818b265bd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:34.044721 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60b41aa-b1e1-4572-a063-547818b265bd None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:34.047918 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 354/717] 10.222.0.46 () {56 vars in 1112 bytes} [Thu May 26 14:55:33 2022] GET /identity/v3/services?name=network => generated 124 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:55:34.048492 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438ed032-1e82-4e81-b43d-8a2ec7058fee None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:34.076577 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:34.139428 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:34.190640 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-438ed032-1e82-4e81-b43d-8a2ec7058fee None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:34.217978 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 364/718] 10.222.0.46 () {58 vars in 1282 bytes} [Thu May 26 14:55:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 479 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 14:55:34.219298 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60b41aa-b1e1-4572-a063-547818b265bd None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:34.229303 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c60c032-e1b3-4c24-bfc4-bbb28c787c54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:34.230021 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c60c032-e1b3-4c24-bfc4-bbb28c787c54 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:34.230307 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c60c032-e1b3-4c24-bfc4-bbb28c787c54 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:34.230587 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c60c032-e1b3-4c24-bfc4-bbb28c787c54 None admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:34.231326 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c60c032-e1b3-4c24-bfc4-bbb28c787c54 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:55:34.231636 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c60c032-e1b3-4c24-bfc4-bbb28c787c54 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:34.233375 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c60c032-e1b3-4c24-bfc4-bbb28c787c54 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:34.252459 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 355/719] 10.222.0.46 () {56 vars in 1109 bytes} [Thu May 26 14:55:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:55:34.253432 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c60c032-e1b3-4c24-bfc4-bbb28c787c54 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:34.349937 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:34.355253 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60b41aa-b1e1-4572-a063-547818b265bd None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:34.379306 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c60c032-e1b3-4c24-bfc4-bbb28c787c54 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:34.395832 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501954b6-f1ba-4c15-bbfa-5c701fbe95cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:34.396685 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501954b6-f1ba-4c15-bbfa-5c701fbe95cc None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:34.397087 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501954b6-f1ba-4c15-bbfa-5c701fbe95cc None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:34.397561 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501954b6-f1ba-4c15-bbfa-5c701fbe95cc None admin] PATH_INFO: `/v3/services` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:34.398740 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501954b6-f1ba-4c15-bbfa-5c701fbe95cc None admin] RBAC: Adding query filter params (type=network) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:55:34.399026 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501954b6-f1ba-4c15-bbfa-5c701fbe95cc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:34.400956 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501954b6-f1ba-4c15-bbfa-5c701fbe95cc None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:34.417494 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 365/720] 10.222.0.46 () {56 vars in 1112 bytes} [Thu May 26 14:55:34 2022] GET /identity/v3/services?type=network => generated 124 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:55:34.417494 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501954b6-f1ba-4c15-bbfa-5c701fbe95cc None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:34.438217 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd55f47-f618-416b-bc5d-c5720d2b209c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:34.439582 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd55f47-f618-416b-bc5d-c5720d2b209c None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:34.439876 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd55f47-f618-416b-bc5d-c5720d2b209c None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:34.440206 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd55f47-f618-416b-bc5d-c5720d2b209c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:34.454867 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:34.456275 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd55f47-f618-416b-bc5d-c5720d2b209c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:34.460100 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd55f47-f618-416b-bc5d-c5720d2b209c None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:34.501972 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-edd55f47-f618-416b-bc5d-c5720d2b209c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 14:55:34.501972 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:55:34.501972 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:55:34.501972 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:55:34.501972 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:55:34.501972 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:55:34.501972 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:55:34.501972 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:55:34.501972 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:55:34.501972 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:55:34.501972 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:55:34.501972 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:55:34.501972 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 14:55:34.501972 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 14:55:34.501972 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 14:55:34.503246 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 14:55:34.503246 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:55:34.503246 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:55:34.503246 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:55:34.503246 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:55:34.503246 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:55:34.503246 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:55:34.503246 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:55:34.503246 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 26 14:55:34.503246 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:55:34.503246 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 26 14:55:34.503246 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:55:34.503246 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:55:34.503246 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:55:34.503246 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 26 14:55:34.503246 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:55:34.503246 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 26 14:55:34.504179 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 14:55:34.504179 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 14:55:34.504179 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 14:55:34.504179 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 26 14:55:34.505363 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 356/721] 10.222.0.46 () {56 vars in 1108 bytes} [Thu May 26 14:55:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:55:34.507549 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e9af68-021f-4730-a463-5781d36e27c8 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:34.507832 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e9af68-021f-4730-a463-5781d36e27c8 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:34.508108 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e9af68-021f-4730-a463-5781d36e27c8 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:34.513885 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 357/722] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 14:55:34 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 14:55:34.515269 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e9af68-021f-4730-a463-5781d36e27c8 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:34.583353 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501954b6-f1ba-4c15-bbfa-5c701fbe95cc None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:34.654743 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ca7692-1bce-47d6-ab9f-847e7d1e463f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:34.655570 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ca7692-1bce-47d6-ab9f-847e7d1e463f None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:34.655869 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ca7692-1bce-47d6-ab9f-847e7d1e463f None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:34.656791 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ca7692-1bce-47d6-ab9f-847e7d1e463f None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:34.666937 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ca7692-1bce-47d6-ab9f-847e7d1e463f None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:34.668031 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ca7692-1bce-47d6-ab9f-847e7d1e463f None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:34.686480 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e9af68-021f-4730-a463-5781d36e27c8 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:34.688451 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-34ca7692-1bce-47d6-ab9f-847e7d1e463f None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 26 14:55:34.688451 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:55:34.688451 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:55:34.688451 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:55:34.688451 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:55:34.688451 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:55:34.688451 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:55:34.688451 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:55:34.688451 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:55:34.688451 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:55:34.688451 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:55:34.688451 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:55:34.688451 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 14:55:34.688451 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 14:55:34.688451 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 14:55:34.688451 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 14:55:34.689886 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:55:34.689886 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:55:34.689886 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 14:55:34.689886 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 14:55:34.689886 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 14:55:34.689886 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 14:55:34.689886 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:55:34.689886 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:55:34.689886 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:55:34.689886 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:55:34.689886 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:55:34.689886 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 26 14:55:34.689886 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:55:34.689886 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 26 14:55:34.689886 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:55:34.689886 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:55:34.689886 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:55:34.689886 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 26 14:55:34.691151 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:55:34.691151 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 26 14:55:34.691151 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 14:55:34.691151 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 14:55:34.691151 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 14:55:34.691151 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 14:55:34.691151 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 14:55:34.691151 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 14:55:34.691151 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 26 14:55:34.691151 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 26 14:55:34.691151 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 366/723] 10.222.0.46 () {56 vars in 1117 bytes} [Thu May 26 14:55:34 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:55:34.691791 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b0f09b-45e1-4f46-9c3e-effdfad08aa8 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:34.691791 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b0f09b-45e1-4f46-9c3e-effdfad08aa8 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:34.692021 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b0f09b-45e1-4f46-9c3e-effdfad08aa8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:34.716281 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-40b0f09b-45e1-4f46-9c3e-effdfad08aa8 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:55:34.717523 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-40b0f09b-45e1-4f46-9c3e-effdfad08aa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:55:34.757937 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:34.766265 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a41608a-b7b7-4733-a0dd-0c2d85718e93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:34.767618 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a41608a-b7b7-4733-a0dd-0c2d85718e93 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:34.768227 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a41608a-b7b7-4733-a0dd-0c2d85718e93 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:34.768825 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a41608a-b7b7-4733-a0dd-0c2d85718e93 None admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:34.770305 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a41608a-b7b7-4733-a0dd-0c2d85718e93 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:55:34.770740 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a41608a-b7b7-4733-a0dd-0c2d85718e93 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:34.772895 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a41608a-b7b7-4733-a0dd-0c2d85718e93 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:34.791929 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 358/724] 10.222.0.46 () {56 vars in 1109 bytes} [Thu May 26 14:55:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:55:34.791929 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a41608a-b7b7-4733-a0dd-0c2d85718e93 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:34.813625 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 367/725] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 14:55:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 14:55:34.814788 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b0f09b-45e1-4f46-9c3e-effdfad08aa8 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:34.832089 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:34.929635 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b0f09b-45e1-4f46-9c3e-effdfad08aa8 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:34.932477 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a41608a-b7b7-4733-a0dd-0c2d85718e93 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:34.970303 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a845ba5a-4633-4a5e-85a8-8185f950bf1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:34.971604 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a845ba5a-4633-4a5e-85a8-8185f950bf1e None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:34.972038 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a845ba5a-4633-4a5e-85a8-8185f950bf1e None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:34.972490 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a845ba5a-4633-4a5e-85a8-8185f950bf1e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:34.979867 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b03cc55-39d5-47e8-bffe-69a1263787c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:34.980558 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b03cc55-39d5-47e8-bffe-69a1263787c8 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:34.980675 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a845ba5a-4633-4a5e-85a8-8185f950bf1e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:34.980844 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b03cc55-39d5-47e8-bffe-69a1263787c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:34.981128 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b03cc55-39d5-47e8-bffe-69a1263787c8 None admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:34.982009 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b03cc55-39d5-47e8-bffe-69a1263787c8 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:55:34.982325 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b03cc55-39d5-47e8-bffe-69a1263787c8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:34.983050 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a845ba5a-4633-4a5e-85a8-8185f950bf1e None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:34.983819 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b03cc55-39d5-47e8-bffe-69a1263787c8 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:34.999336 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-a845ba5a-4633-4a5e-85a8-8185f950bf1e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 14:55:34.999336 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:55:34.999336 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:55:34.999336 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:55:34.999336 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:55:34.999336 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:55:34.999336 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:55:34.999336 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:55:34.999336 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:55:34.999336 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:55:34.999336 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:55:34.999336 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:55:34.999336 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 14:55:34.999336 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 14:55:34.999336 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 14:55:35.000393 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 14:55:35.000393 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:55:35.000393 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:55:35.000393 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:55:35.000393 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:55:35.000393 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:55:35.000393 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:55:35.000393 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:55:35.000393 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 26 14:55:35.000393 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:55:35.000393 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 26 14:55:35.000393 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:55:35.000393 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:55:35.000393 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:55:35.000393 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 26 14:55:35.000393 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:55:35.000393 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 26 14:55:35.001394 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 14:55:35.001394 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 14:55:35.001394 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 14:55:35.001394 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 14:55:35.001394 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 14:55:35.001394 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 14:55:35.001394 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 14:55:35.001394 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 26 14:55:35.003773 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 368/726] 10.222.0.46 () {56 vars in 1111 bytes} [Thu May 26 14:55:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:55:35.005190 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a845ba5a-4633-4a5e-85a8-8185f950bf1e None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:35.005607 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 359/727] 10.222.0.46 () {56 vars in 1149 bytes} [Thu May 26 14:55:34 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:55:35.012347 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b03cc55-39d5-47e8-bffe-69a1263787c8 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:35.071792 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:35.092218 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:35.129401 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a845ba5a-4633-4a5e-85a8-8185f950bf1e None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:35.138229 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b03cc55-39d5-47e8-bffe-69a1263787c8 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:35.173688 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033b0a8c-f100-4951-95e7-49644668f4c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:35.174745 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033b0a8c-f100-4951-95e7-49644668f4c3 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:35.175424 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033b0a8c-f100-4951-95e7-49644668f4c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:35.175714 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033b0a8c-f100-4951-95e7-49644668f4c3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:35.180278 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79288dab-dbde-4d00-a77b-5ddd9e8fc6c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:35.181122 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79288dab-dbde-4d00-a77b-5ddd9e8fc6c5 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:35.181581 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79288dab-dbde-4d00-a77b-5ddd9e8fc6c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:35.182588 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79288dab-dbde-4d00-a77b-5ddd9e8fc6c5 None admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:35.183922 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79288dab-dbde-4d00-a77b-5ddd9e8fc6c5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:55:35.184389 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79288dab-dbde-4d00-a77b-5ddd9e8fc6c5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:35.186027 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79288dab-dbde-4d00-a77b-5ddd9e8fc6c5 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:35.188440 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033b0a8c-f100-4951-95e7-49644668f4c3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 14:55:35.198739 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-033b0a8c-f100-4951-95e7-49644668f4c3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 14:55:35.198739 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:55:35.198739 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:55:35.198739 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:55:35.198739 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:55:35.198739 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:55:35.198739 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:55:35.198739 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:55:35.198739 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:55:35.198739 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:55:35.198739 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:55:35.198739 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:55:35.198739 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 14:55:35.198739 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 14:55:35.198739 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 14:55:35.198739 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 14:55:35.199967 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 14:55:35.199967 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 14:55:35.199967 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 14:55:35.199967 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 14:55:35.199967 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:55:35.199967 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:55:35.199967 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:55:35.199967 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:55:35.199967 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:55:35.199967 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:55:35.199967 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:55:35.199967 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 26 14:55:35.199967 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:55:35.199967 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 26 14:55:35.199967 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:55:35.199967 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:55:35.201182 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:55:35.201182 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 26 14:55:35.201182 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:55:35.201182 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 26 14:55:35.201182 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 14:55:35.201182 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 14:55:35.201182 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 14:55:35.201182 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 14:55:35.201182 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 14:55:35.201182 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 14:55:35.201182 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 14:55:35.201182 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 26 14:55:35.201182 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 369/728] 10.222.0.46 () {56 vars in 1096 bytes} [Thu May 26 14:55:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:55:35.201182 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033b0a8c-f100-4951-95e7-49644668f4c3 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:35.203388 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 360/729] 10.222.0.46 () {56 vars in 1148 bytes} [Thu May 26 14:55:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:55:35.206281 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79288dab-dbde-4d00-a77b-5ddd9e8fc6c5 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:35.324867 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79288dab-dbde-4d00-a77b-5ddd9e8fc6c5 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:35.337124 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:35.356805 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033b0a8c-f100-4951-95e7-49644668f4c3 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:35.376120 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364e3997-2f59-4ac9-bfcc-2b9477d31bd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:35.377250 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364e3997-2f59-4ac9-bfcc-2b9477d31bd7 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:35.377558 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364e3997-2f59-4ac9-bfcc-2b9477d31bd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:35.377836 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364e3997-2f59-4ac9-bfcc-2b9477d31bd7 None admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:35.378593 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364e3997-2f59-4ac9-bfcc-2b9477d31bd7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:55:35.378899 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364e3997-2f59-4ac9-bfcc-2b9477d31bd7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:35.380550 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364e3997-2f59-4ac9-bfcc-2b9477d31bd7 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:35.393861 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 361/730] 10.222.0.46 () {56 vars in 1099 bytes} [Thu May 26 14:55:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 14:55:35.394818 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364e3997-2f59-4ac9-bfcc-2b9477d31bd7 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:35.400613 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c661ffc9-523b-4da1-8ab3-bee5f98b2557 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:35.401554 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c661ffc9-523b-4da1-8ab3-bee5f98b2557 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:35.402043 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c661ffc9-523b-4da1-8ab3-bee5f98b2557 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:35.402563 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c661ffc9-523b-4da1-8ab3-bee5f98b2557 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:35.405971 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:35.481136 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c661ffc9-523b-4da1-8ab3-bee5f98b2557 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:35.483034 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c661ffc9-523b-4da1-8ab3-bee5f98b2557 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:35.531836 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364e3997-2f59-4ac9-bfcc-2b9477d31bd7 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:35.568664 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3930948-8d0a-416c-9eff-c20f7a7493df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:35.569438 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3930948-8d0a-416c-9eff-c20f7a7493df None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:35.569674 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3930948-8d0a-416c-9eff-c20f7a7493df None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:35.570022 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3930948-8d0a-416c-9eff-c20f7a7493df None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:35.570820 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3930948-8d0a-416c-9eff-c20f7a7493df None admin] RBAC: Adding query filter params (role.id=a2e551773cbd4df084020676865b8222, scope.project.id=a17fc8227d4c4da6b2847b350e912ee3, user.id=d6639353c70449d1b3d10cd6487c3b00) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:55:35.571107 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3930948-8d0a-416c-9eff-c20f7a7493df None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:35.572809 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3930948-8d0a-416c-9eff-c20f7a7493df None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:35.589500 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 362/731] 10.222.0.46 () {56 vars in 1374 bytes} [Thu May 26 14:55:35 2022] GET /identity/v3/role_assignments?user.id=d6639353c70449d1b3d10cd6487c3b00&scope.project.id=a17fc8227d4c4da6b2847b350e912ee3&role.id=a2e551773cbd4df084020676865b8222 => generated 259 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:55:35.589500 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3930948-8d0a-416c-9eff-c20f7a7493df None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:35.633754 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 370/732] 10.222.0.46 () {58 vars in 1282 bytes} [Thu May 26 14:55:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 26 14:55:35.635820 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4ff68d-1e16-4e3b-8f0b-040a4195741a None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:35.636269 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4ff68d-1e16-4e3b-8f0b-040a4195741a None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:35.636690 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4ff68d-1e16-4e3b-8f0b-040a4195741a None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:35.642829 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 371/733] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 14:55:35 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 14:55:35.644085 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4ff68d-1e16-4e3b-8f0b-040a4195741a None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:35.645351 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:35.703287 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:35.724742 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3930948-8d0a-416c-9eff-c20f7a7493df None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:35.774536 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b120a1dc-b6a7-4308-aad0-31d4aeefd06b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:35.776163 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b120a1dc-b6a7-4308-aad0-31d4aeefd06b None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:35.776915 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b120a1dc-b6a7-4308-aad0-31d4aeefd06b None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:35.777342 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b120a1dc-b6a7-4308-aad0-31d4aeefd06b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:35.792858 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b120a1dc-b6a7-4308-aad0-31d4aeefd06b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:35.794757 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b120a1dc-b6a7-4308-aad0-31d4aeefd06b None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:35.806449 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4ff68d-1e16-4e3b-8f0b-040a4195741a None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:35.815478 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-b120a1dc-b6a7-4308-aad0-31d4aeefd06b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 14:55:35.815478 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:55:35.815478 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:55:35.815478 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:55:35.815478 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:55:35.815478 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:55:35.815478 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:55:35.815478 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:55:35.815478 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:55:35.815478 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:55:35.815478 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:55:35.815478 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:55:35.815478 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 14:55:35.815478 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 14:55:35.815478 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 14:55:35.816451 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 14:55:35.816451 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:55:35.816451 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:55:35.816451 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:55:35.816451 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:55:35.816451 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:55:35.816451 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:55:35.816451 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:55:35.816451 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 26 14:55:35.816451 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:55:35.816451 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 26 14:55:35.816451 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:55:35.816451 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:55:35.816451 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:55:35.816451 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 26 14:55:35.816451 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:55:35.816451 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 26 14:55:35.817346 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 14:55:35.817346 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 14:55:35.817346 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 14:55:35.817346 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 26 14:55:35.817346 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 363/734] 10.222.0.46 () {56 vars in 1108 bytes} [Thu May 26 14:55:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:55:35.818421 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e914e2f-41f3-42af-81e5-4a9bf0f698d2 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:35.818664 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e914e2f-41f3-42af-81e5-4a9bf0f698d2 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:35.819735 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e914e2f-41f3-42af-81e5-4a9bf0f698d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:35.837171 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-9e914e2f-41f3-42af-81e5-4a9bf0f698d2 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:55:35.838969 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-9e914e2f-41f3-42af-81e5-4a9bf0f698d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:55:35.845398 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccb8742-8789-4db0-8df9-1bd9580dfcda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:35.846324 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccb8742-8789-4db0-8df9-1bd9580dfcda None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:35.846603 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccb8742-8789-4db0-8df9-1bd9580dfcda None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:35.846876 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ccb8742-8789-4db0-8df9-1bd9580dfcda None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:35.858250 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ccb8742-8789-4db0-8df9-1bd9580dfcda None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:35.859843 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ccb8742-8789-4db0-8df9-1bd9580dfcda None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:35.880175 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-1ccb8742-8789-4db0-8df9-1bd9580dfcda None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 14:55:35.880175 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:55:35.880175 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:55:35.880175 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:55:35.880175 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:55:35.880175 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:55:35.880175 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:55:35.880175 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:55:35.880175 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:55:35.880175 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:55:35.880175 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:55:35.880175 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:55:35.880175 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 14:55:35.880175 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 14:55:35.880175 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 14:55:35.881160 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 14:55:35.881160 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:55:35.881160 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:55:35.881160 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:55:35.881160 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:55:35.881160 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:55:35.881160 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:55:35.881160 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:55:35.881160 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 26 14:55:35.881160 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:55:35.881160 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 26 14:55:35.881160 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:55:35.881160 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:55:35.881160 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:55:35.881160 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 26 14:55:35.881160 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:55:35.881160 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 26 14:55:35.882101 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 14:55:35.882101 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 14:55:35.882101 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 14:55:35.882101 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 26 14:55:35.882101 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 372/735] 10.222.0.46 () {56 vars in 1108 bytes} [Thu May 26 14:55:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:55:35.882843 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccb8742-8789-4db0-8df9-1bd9580dfcda None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:35.932659 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 364/736] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 14:55:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 14:55:35.933669 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e914e2f-41f3-42af-81e5-4a9bf0f698d2 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:35.956315 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:35.997495 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:36.016025 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ccb8742-8789-4db0-8df9-1bd9580dfcda None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:36.053422 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfc0bbc-2196-4610-af43-72e989806c6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:36.054384 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfc0bbc-2196-4610-af43-72e989806c6c None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:36.054820 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfc0bbc-2196-4610-af43-72e989806c6c None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:36.055244 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfc0bbc-2196-4610-af43-72e989806c6c None admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:36.056122 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfc0bbc-2196-4610-af43-72e989806c6c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:55:36.056578 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfc0bbc-2196-4610-af43-72e989806c6c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:36.057705 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e914e2f-41f3-42af-81e5-4a9bf0f698d2 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:36.058540 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfc0bbc-2196-4610-af43-72e989806c6c None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:36.071775 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 373/737] 10.222.0.46 () {56 vars in 1109 bytes} [Thu May 26 14:55:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:55:36.073839 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfc0bbc-2196-4610-af43-72e989806c6c None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:36.110154 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee2cd62-3d72-42f8-9cf1-20941e9510b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:36.110839 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee2cd62-3d72-42f8-9cf1-20941e9510b3 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:36.111120 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee2cd62-3d72-42f8-9cf1-20941e9510b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:36.111391 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee2cd62-3d72-42f8-9cf1-20941e9510b3 None admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:36.112302 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee2cd62-3d72-42f8-9cf1-20941e9510b3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:55:36.112302 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee2cd62-3d72-42f8-9cf1-20941e9510b3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:36.113827 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee2cd62-3d72-42f8-9cf1-20941e9510b3 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:36.131235 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 365/738] 10.222.0.46 () {56 vars in 1109 bytes} [Thu May 26 14:55:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:55:36.131235 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee2cd62-3d72-42f8-9cf1-20941e9510b3 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:36.217607 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfc0bbc-2196-4610-af43-72e989806c6c None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:36.250510 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:36.256279 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb17e94b-980e-4c22-9ab3-1acf9ad07dc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:36.257211 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb17e94b-980e-4c22-9ab3-1acf9ad07dc1 None admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:36.257792 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb17e94b-980e-4c22-9ab3-1acf9ad07dc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:36.258232 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb17e94b-980e-4c22-9ab3-1acf9ad07dc1 None admin] PATH_INFO: `/v3/services` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:36.259204 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb17e94b-980e-4c22-9ab3-1acf9ad07dc1 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:36.261294 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb17e94b-980e-4c22-9ab3-1acf9ad07dc1 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:36.279954 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee2cd62-3d72-42f8-9cf1-20941e9510b3 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:36.290166 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 374/739] 10.222.0.46 () {60 vars in 1141 bytes} [Thu May 26 14:55:36 2022] POST /identity/v3/services => generated 251 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 14:55:36.290747 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb17e94b-980e-4c22-9ab3-1acf9ad07dc1 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:36.319970 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:36.401829 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e94633-ca67-4aa7-82ed-712a1d3a2862 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:36.402885 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e94633-ca67-4aa7-82ed-712a1d3a2862 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:36.403163 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e94633-ca67-4aa7-82ed-712a1d3a2862 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:36.403433 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e94633-ca67-4aa7-82ed-712a1d3a2862 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:36.420723 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e94633-ca67-4aa7-82ed-712a1d3a2862 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:36.424752 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e94633-ca67-4aa7-82ed-712a1d3a2862 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:36.455350 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-11e94633-ca67-4aa7-82ed-712a1d3a2862 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 26 14:55:36.455350 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:55:36.455350 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:55:36.455350 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:55:36.455350 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:55:36.455350 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:55:36.455350 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:55:36.455350 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:55:36.455350 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:55:36.455350 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:55:36.455350 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:55:36.455350 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:55:36.455350 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 14:55:36.455350 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 14:55:36.455350 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 14:55:36.455350 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 14:55:36.456394 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:55:36.456394 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:55:36.456394 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 14:55:36.456394 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 14:55:36.456394 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 14:55:36.456394 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 14:55:36.456394 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:55:36.456394 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:55:36.456394 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:55:36.456394 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:55:36.456394 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:55:36.456394 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 26 14:55:36.456394 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:55:36.456394 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 26 14:55:36.456394 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:55:36.456394 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:55:36.456394 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:55:36.456394 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 26 14:55:36.457553 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:55:36.457553 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 26 14:55:36.457553 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 14:55:36.457553 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 14:55:36.457553 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 14:55:36.457553 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 14:55:36.457553 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 14:55:36.457553 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 14:55:36.457553 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 26 14:55:36.457553 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 26 14:55:36.457553 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 366/740] 10.222.0.46 () {56 vars in 1096 bytes} [Thu May 26 14:55:36 2022] GET /identity/v3/users/swift => generated 83 bytes in 329 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:55:36.462527 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e94633-ca67-4aa7-82ed-712a1d3a2862 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:36.484788 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb17e94b-980e-4c22-9ab3-1acf9ad07dc1 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:36.532552 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26295ad6-aa21-4f2c-96fd-236cd8bd4fbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:36.532787 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26295ad6-aa21-4f2c-96fd-236cd8bd4fbd None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:36.533031 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26295ad6-aa21-4f2c-96fd-236cd8bd4fbd None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:36.533316 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26295ad6-aa21-4f2c-96fd-236cd8bd4fbd None admin] PATH_INFO: `/v3/projects/service` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:36.548612 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26295ad6-aa21-4f2c-96fd-236cd8bd4fbd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:36.548612 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26295ad6-aa21-4f2c-96fd-236cd8bd4fbd None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:36.577412 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-26295ad6-aa21-4f2c-96fd-236cd8bd4fbd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 14:55:36.577412 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:55:36.577412 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:55:36.577412 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:55:36.577412 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:55:36.577412 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:55:36.577412 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:55:36.577412 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:55:36.577412 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:55:36.577412 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:55:36.577412 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:55:36.577412 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:55:36.577412 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 14:55:36.577412 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 14:55:36.577412 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 14:55:36.580436 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 14:55:36.580436 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:55:36.580436 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:55:36.580436 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:55:36.580436 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:55:36.580436 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:55:36.580436 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:55:36.580436 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:55:36.580436 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 26 14:55:36.580436 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:55:36.580436 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 26 14:55:36.580436 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:55:36.580436 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:55:36.580436 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:55:36.580436 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 26 14:55:36.580436 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:55:36.580436 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 26 14:55:36.582682 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 14:55:36.582682 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 14:55:36.582682 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 14:55:36.582682 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 14:55:36.582682 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 14:55:36.582682 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 14:55:36.582682 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 14:55:36.582682 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 26 14:55:36.582682 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 375/741] 10.222.0.46 () {56 vars in 1111 bytes} [Thu May 26 14:55:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:55:36.584140 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26295ad6-aa21-4f2c-96fd-236cd8bd4fbd None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:36.670575 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e94633-ca67-4aa7-82ed-712a1d3a2862 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:36.698478 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:36.712465 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cee0f2-0f77-4ba9-8f0b-4be8f6a6b160 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:36.713171 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cee0f2-0f77-4ba9-8f0b-4be8f6a6b160 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:36.713469 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cee0f2-0f77-4ba9-8f0b-4be8f6a6b160 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:36.713759 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cee0f2-0f77-4ba9-8f0b-4be8f6a6b160 None admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:36.714598 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94cee0f2-0f77-4ba9-8f0b-4be8f6a6b160 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:55:36.714916 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94cee0f2-0f77-4ba9-8f0b-4be8f6a6b160 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:36.716350 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94cee0f2-0f77-4ba9-8f0b-4be8f6a6b160 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:36.718727 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:36.735982 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 367/742] 10.222.0.46 () {56 vars in 1135 bytes} [Thu May 26 14:55:36 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:55:36.741020 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cee0f2-0f77-4ba9-8f0b-4be8f6a6b160 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:36.755665 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26295ad6-aa21-4f2c-96fd-236cd8bd4fbd None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:36.829165 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ebc8c2-c9ee-44fa-8164-16ead3f49d8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:36.830107 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ebc8c2-c9ee-44fa-8164-16ead3f49d8e None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:36.830538 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ebc8c2-c9ee-44fa-8164-16ead3f49d8e None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:36.830962 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ebc8c2-c9ee-44fa-8164-16ead3f49d8e None admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:36.831929 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ebc8c2-c9ee-44fa-8164-16ead3f49d8e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:55:36.832385 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ebc8c2-c9ee-44fa-8164-16ead3f49d8e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:36.833995 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ebc8c2-c9ee-44fa-8164-16ead3f49d8e None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:36.865631 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 376/743] 10.222.0.46 () {56 vars in 1148 bytes} [Thu May 26 14:55:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 14:55:36.866839 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179eaf14-7054-4a73-a901-61b569eaabf7 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:36.867131 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179eaf14-7054-4a73-a901-61b569eaabf7 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:36.867407 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179eaf14-7054-4a73-a901-61b569eaabf7 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:36.868894 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 377/744] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 14:55:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 14:55:36.874146 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179eaf14-7054-4a73-a901-61b569eaabf7 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:36.899082 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cee0f2-0f77-4ba9-8f0b-4be8f6a6b160 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:36.936192 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7dc2511-b6ed-49b6-b3e8-518c49451ee8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:36.937131 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dc2511-b6ed-49b6-b3e8-518c49451ee8 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:36.937609 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dc2511-b6ed-49b6-b3e8-518c49451ee8 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:36.938032 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7dc2511-b6ed-49b6-b3e8-518c49451ee8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:36.994100 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179eaf14-7054-4a73-a901-61b569eaabf7 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:37.004870 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:37.016642 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7dc2511-b6ed-49b6-b3e8-518c49451ee8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:37.018585 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7dc2511-b6ed-49b6-b3e8-518c49451ee8 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:37.031732 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:37.043578 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796229d0-db2f-4ef7-a8dd-e2ff96b13d5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:37.044412 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796229d0-db2f-4ef7-a8dd-e2ff96b13d5f None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:37.044837 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796229d0-db2f-4ef7-a8dd-e2ff96b13d5f None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:37.045246 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796229d0-db2f-4ef7-a8dd-e2ff96b13d5f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:37.046351 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796229d0-db2f-4ef7-a8dd-e2ff96b13d5f None admin] RBAC: Adding query filter params (role.id=eb76213820fa4c4e9b0a5d5ac2a08b41, scope.project.id=a17fc8227d4c4da6b2847b350e912ee3, user.id=8028c520d93449a6a087c281280ee13d) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:55:37.047857 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796229d0-db2f-4ef7-a8dd-e2ff96b13d5f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:37.049646 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796229d0-db2f-4ef7-a8dd-e2ff96b13d5f None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:37.057531 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 378/745] 10.222.0.46 () {56 vars in 1374 bytes} [Thu May 26 14:55:36 2022] GET /identity/v3/role_assignments?user.id=8028c520d93449a6a087c281280ee13d&scope.project.id=a17fc8227d4c4da6b2847b350e912ee3&role.id=eb76213820fa4c4e9b0a5d5ac2a08b41 => generated 259 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:55:37.059903 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a2a9c5-d1e0-464e-b6d1-3ab234ed28ee None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:37.060361 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a2a9c5-d1e0-464e-b6d1-3ab234ed28ee None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:37.060792 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a2a9c5-d1e0-464e-b6d1-3ab234ed28ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:37.083594 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-a0a2a9c5-d1e0-464e-b6d1-3ab234ed28ee None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:55:37.086016 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-a0a2a9c5-d1e0-464e-b6d1-3ab234ed28ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:55:37.155207 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 368/746] 10.222.0.46 () {58 vars in 1282 bytes} [Thu May 26 14:55:36 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 26 14:55:37.160664 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7dc2511-b6ed-49b6-b3e8-518c49451ee8 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:37.242783 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 379/747] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 14:55:37 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 26 14:55:37.259928 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a2a9c5-d1e0-464e-b6d1-3ab234ed28ee None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:37.317891 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:37.329900 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7dc2511-b6ed-49b6-b3e8-518c49451ee8 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:37.402281 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a2a9c5-d1e0-464e-b6d1-3ab234ed28ee None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:37.407669 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a046373-b7b9-4a86-b40c-e8148c7c0c49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:37.408631 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a046373-b7b9-4a86-b40c-e8148c7c0c49 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:37.408921 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a046373-b7b9-4a86-b40c-e8148c7c0c49 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:37.409286 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a046373-b7b9-4a86-b40c-e8148c7c0c49 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:37.429253 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:37.435761 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a046373-b7b9-4a86-b40c-e8148c7c0c49 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:37.437239 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a046373-b7b9-4a86-b40c-e8148c7c0c49 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:37.446189 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f571c88-7c75-4caf-ae75-0087bcd38db0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:37.446866 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f571c88-7c75-4caf-ae75-0087bcd38db0 None admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:37.447152 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f571c88-7c75-4caf-ae75-0087bcd38db0 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:37.447970 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f571c88-7c75-4caf-ae75-0087bcd38db0 None admin] PATH_INFO: `/v3/services` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:37.448706 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f571c88-7c75-4caf-ae75-0087bcd38db0 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:37.450202 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f571c88-7c75-4caf-ae75-0087bcd38db0 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:37.469080 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 380/748] 10.222.0.46 () {60 vars in 1141 bytes} [Thu May 26 14:55:37 2022] POST /identity/v3/services => generated 240 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 14:55:37.472722 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-4a046373-b7b9-4a86-b40c-e8148c7c0c49 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 14:55:37.472722 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:55:37.472722 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:55:37.472722 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:55:37.472722 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:55:37.472722 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:55:37.472722 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:55:37.472722 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:55:37.472722 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:55:37.472722 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:55:37.472722 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:55:37.472722 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:55:37.472722 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 14:55:37.472722 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 14:55:37.472722 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 14:55:37.473681 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 14:55:37.473681 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:55:37.473681 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:55:37.473681 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:55:37.473681 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:55:37.473681 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:55:37.473681 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:55:37.473681 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:55:37.473681 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 26 14:55:37.473681 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:55:37.473681 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 26 14:55:37.473681 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:55:37.473681 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:55:37.473681 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:55:37.473681 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 26 14:55:37.473681 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:55:37.473681 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 26 14:55:37.474798 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 14:55:37.474798 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 14:55:37.474798 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 14:55:37.474798 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 26 14:55:37.474798 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 369/749] 10.222.0.46 () {56 vars in 1108 bytes} [Thu May 26 14:55:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 314 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:55:37.480016 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f571c88-7c75-4caf-ae75-0087bcd38db0 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:37.650969 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f571c88-7c75-4caf-ae75-0087bcd38db0 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:37.703218 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd8b142-c019-4ffd-8be8-90579cdc6aff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:37.703827 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd8b142-c019-4ffd-8be8-90579cdc6aff None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:37.704102 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd8b142-c019-4ffd-8be8-90579cdc6aff None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:37.704382 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd8b142-c019-4ffd-8be8-90579cdc6aff None admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:37.705125 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd8b142-c019-4ffd-8be8-90579cdc6aff None admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:55:37.705433 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd8b142-c019-4ffd-8be8-90579cdc6aff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:37.706868 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd8b142-c019-4ffd-8be8-90579cdc6aff None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:37.722079 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 381/750] 10.222.0.46 () {56 vars in 1109 bytes} [Thu May 26 14:55:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:55:37.727222 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a046373-b7b9-4a86-b40c-e8148c7c0c49 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:37.898322 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a046373-b7b9-4a86-b40c-e8148c7c0c49 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:37.949341 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060b4a5b-af52-45e0-91ce-2343dde3fdd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:37.950803 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060b4a5b-af52-45e0-91ce-2343dde3fdd8 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:37.951488 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060b4a5b-af52-45e0-91ce-2343dde3fdd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:37.952169 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060b4a5b-af52-45e0-91ce-2343dde3fdd8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:37.966633 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060b4a5b-af52-45e0-91ce-2343dde3fdd8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:37.968491 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060b4a5b-af52-45e0-91ce-2343dde3fdd8 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:37.988882 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-060b4a5b-af52-45e0-91ce-2343dde3fdd8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 14:55:37.988882 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:55:37.988882 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:55:37.988882 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:55:37.988882 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:55:37.988882 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:55:37.988882 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:55:37.988882 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:55:37.988882 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:55:37.988882 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:55:37.988882 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:55:37.988882 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:55:37.988882 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 14:55:37.988882 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 14:55:37.988882 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 14:55:37.989890 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 14:55:37.989890 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:55:37.989890 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:55:37.989890 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:55:37.989890 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:55:37.989890 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:55:37.989890 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:55:37.989890 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:55:37.989890 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 26 14:55:37.989890 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:55:37.989890 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 26 14:55:37.989890 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:55:37.989890 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:55:37.989890 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:55:37.989890 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 26 14:55:37.989890 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:55:37.989890 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 26 14:55:37.990781 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 14:55:37.990781 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 14:55:37.990781 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 14:55:37.990781 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 14:55:37.990781 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 14:55:37.990781 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 14:55:37.990781 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 14:55:37.990781 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 26 14:55:37.993362 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 370/751] 10.222.0.46 () {56 vars in 1111 bytes} [Thu May 26 14:55:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:55:38.000182 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd8b142-c019-4ffd-8be8-90579cdc6aff None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:38.051943 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:38.150090 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd8b142-c019-4ffd-8be8-90579cdc6aff None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:38.207579 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425ba404-42ca-4eab-be2a-0e70516157c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:38.208310 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425ba404-42ca-4eab-be2a-0e70516157c6 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:38.208582 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7d0257-578c-4dc9-bb74-30d5ed1630ce None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:38.208690 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425ba404-42ca-4eab-be2a-0e70516157c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:38.208872 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7d0257-578c-4dc9-bb74-30d5ed1630ce None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:38.208972 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425ba404-42ca-4eab-be2a-0e70516157c6 None admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:38.209154 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7d0257-578c-4dc9-bb74-30d5ed1630ce None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:38.209695 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425ba404-42ca-4eab-be2a-0e70516157c6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:55:38.210010 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425ba404-42ca-4eab-be2a-0e70516157c6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:38.211450 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425ba404-42ca-4eab-be2a-0e70516157c6 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:38.212613 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 371/752] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 14:55:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 14:55:38.222359 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa6778e-2699-4c50-9c01-5e8bf0158314 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:38.222630 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa6778e-2699-4c50-9c01-5e8bf0158314 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:38.222859 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa6778e-2699-4c50-9c01-5e8bf0158314 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:38.225391 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 382/753] 10.222.0.46 () {56 vars in 1148 bytes} [Thu May 26 14:55:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:55:38.231278 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425ba404-42ca-4eab-be2a-0e70516157c6 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:38.241036 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:38.266603 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-2aa6778e-2699-4c50-9c01-5e8bf0158314 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:55:38.267880 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-2aa6778e-2699-4c50-9c01-5e8bf0158314 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:55:38.347250 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425ba404-42ca-4eab-be2a-0e70516157c6 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:38.357894 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:38.388249 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a98d21f-7181-4822-b839-44065204ef92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:38.389300 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a98d21f-7181-4822-b839-44065204ef92 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:38.389629 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a98d21f-7181-4822-b839-44065204ef92 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:38.389916 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a98d21f-7181-4822-b839-44065204ef92 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:38.390931 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a98d21f-7181-4822-b839-44065204ef92 None admin] RBAC: Adding query filter params (role.id=a2e551773cbd4df084020676865b8222, scope.project.id=a17fc8227d4c4da6b2847b350e912ee3, user.id=a4bea272624b45d8ac2ad27bd07df885) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:55:38.391261 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a98d21f-7181-4822-b839-44065204ef92 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:38.392708 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a98d21f-7181-4822-b839-44065204ef92 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:38.399099 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 383/754] 10.222.0.46 () {56 vars in 1374 bytes} [Thu May 26 14:55:38 2022] GET /identity/v3/role_assignments?user.id=a4bea272624b45d8ac2ad27bd07df885&scope.project.id=a17fc8227d4c4da6b2847b350e912ee3&role.id=a2e551773cbd4df084020676865b8222 => generated 259 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 14:55:38.491290 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 372/755] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 14:55:38 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 270 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 14:55:38.535609 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a98d21f-7181-4822-b839-44065204ef92 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:38.678687 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a98d21f-7181-4822-b839-44065204ef92 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:38.751921 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddeffe52-0bb2-4c9d-8951-54991386d426 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:38.752689 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddeffe52-0bb2-4c9d-8951-54991386d426 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:38.752971 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddeffe52-0bb2-4c9d-8951-54991386d426 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:38.753248 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddeffe52-0bb2-4c9d-8951-54991386d426 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:38.779146 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddeffe52-0bb2-4c9d-8951-54991386d426 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 14:55:38.791585 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-ddeffe52-0bb2-4c9d-8951-54991386d426 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 14:55:38.791585 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:55:38.791585 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:55:38.791585 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:55:38.791585 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:55:38.791585 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:55:38.791585 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:55:38.791585 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:55:38.791585 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:55:38.791585 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:55:38.791585 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:55:38.791585 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:55:38.791585 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 14:55:38.791585 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 14:55:38.791585 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 14:55:38.791585 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 14:55:38.792609 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 14:55:38.792609 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 14:55:38.792609 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 14:55:38.792609 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 14:55:38.792609 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:55:38.792609 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:55:38.792609 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:55:38.792609 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:55:38.792609 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:55:38.792609 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:55:38.792609 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:55:38.792609 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 26 14:55:38.792609 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:55:38.792609 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 26 14:55:38.792609 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:55:38.792609 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:55:38.793499 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:55:38.793499 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 26 14:55:38.793499 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:55:38.793499 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 26 14:55:38.793499 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 14:55:38.793499 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 14:55:38.793499 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 14:55:38.793499 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 14:55:38.793499 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 14:55:38.793499 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 14:55:38.793499 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 14:55:38.793499 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 26 14:55:38.793499 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 384/756] 10.222.0.46 () {56 vars in 1096 bytes} [Thu May 26 14:55:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:55:38.800495 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa6778e-2699-4c50-9c01-5e8bf0158314 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:38.969770 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa6778e-2699-4c50-9c01-5e8bf0158314 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:39.021238 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db872cf8-6b42-4c26-b6ae-296d80eb8fb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:39.022009 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db872cf8-6b42-4c26-b6ae-296d80eb8fb8 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:39.024404 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db872cf8-6b42-4c26-b6ae-296d80eb8fb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:39.024404 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db872cf8-6b42-4c26-b6ae-296d80eb8fb8 None admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:39.024404 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db872cf8-6b42-4c26-b6ae-296d80eb8fb8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:55:39.024404 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db872cf8-6b42-4c26-b6ae-296d80eb8fb8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:39.028931 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db872cf8-6b42-4c26-b6ae-296d80eb8fb8 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:39.028931 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:39.048065 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 373/757] 10.222.0.46 () {56 vars in 1099 bytes} [Thu May 26 14:55:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:55:39.053825 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddeffe52-0bb2-4c9d-8951-54991386d426 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:39.107060 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:39.238689 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddeffe52-0bb2-4c9d-8951-54991386d426 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:39.276982 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3056e5c-79be-443c-a0be-efc620d7fb22 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:39.277606 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3056e5c-79be-443c-a0be-efc620d7fb22 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:39.278084 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3056e5c-79be-443c-a0be-efc620d7fb22 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:39.279915 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 374/758] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 14:55:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 14:55:39.285364 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca13f5b-5eb6-438d-bfac-08569886c90a None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:39.285991 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca13f5b-5eb6-438d-bfac-08569886c90a None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:39.286486 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca13f5b-5eb6-438d-bfac-08569886c90a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:39.292742 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235545e9-060d-45f0-8ed8-24043be71572 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:39.295008 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235545e9-060d-45f0-8ed8-24043be71572 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:39.295008 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235545e9-060d-45f0-8ed8-24043be71572 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:39.295008 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235545e9-060d-45f0-8ed8-24043be71572 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:39.316510 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235545e9-060d-45f0-8ed8-24043be71572 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:39.317763 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-8ca13f5b-5eb6-438d-bfac-08569886c90a None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:55:39.319394 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235545e9-060d-45f0-8ed8-24043be71572 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:39.320802 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-8ca13f5b-5eb6-438d-bfac-08569886c90a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:55:39.348413 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-235545e9-060d-45f0-8ed8-24043be71572 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 14:55:39.348413 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:55:39.348413 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:55:39.348413 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:55:39.348413 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:55:39.348413 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:55:39.348413 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:55:39.348413 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:55:39.348413 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:55:39.348413 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:55:39.348413 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:55:39.348413 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:55:39.348413 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 14:55:39.348413 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 14:55:39.348413 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 14:55:39.349565 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 14:55:39.349565 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:55:39.349565 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:55:39.349565 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:55:39.349565 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:55:39.349565 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:55:39.349565 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:55:39.349565 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:55:39.349565 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 26 14:55:39.349565 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:55:39.349565 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 26 14:55:39.349565 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:55:39.349565 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:55:39.349565 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:55:39.349565 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 26 14:55:39.349565 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:55:39.349565 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 26 14:55:39.350671 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 14:55:39.350671 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 14:55:39.350671 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 14:55:39.350671 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 26 14:55:39.352766 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 385/759] 10.222.0.46 () {56 vars in 1108 bytes} [Thu May 26 14:55:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:55:39.366259 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235545e9-060d-45f0-8ed8-24043be71572 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:39.463818 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 375/760] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 14:55:39 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 26 14:55:39.508923 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:39.510857 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca13f5b-5eb6-438d-bfac-08569886c90a None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:39.568813 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235545e9-060d-45f0-8ed8-24043be71572 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:39.654781 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c4f57e-47a0-4c31-839a-d2c2c044a7de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:39.655469 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c4f57e-47a0-4c31-839a-d2c2c044a7de None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:39.655755 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c4f57e-47a0-4c31-839a-d2c2c044a7de None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:39.656077 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c4f57e-47a0-4c31-839a-d2c2c044a7de None admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:39.656822 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c4f57e-47a0-4c31-839a-d2c2c044a7de None admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:55:39.657170 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c4f57e-47a0-4c31-839a-d2c2c044a7de None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:39.658671 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c4f57e-47a0-4c31-839a-d2c2c044a7de None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:39.686772 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca13f5b-5eb6-438d-bfac-08569886c90a None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:39.691982 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 386/761] 10.222.0.46 () {56 vars in 1109 bytes} [Thu May 26 14:55:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:55:39.698411 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c4f57e-47a0-4c31-839a-d2c2c044a7de None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:39.726014 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:39.742190 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a517bbe3-eda4-48b0-92f7-416850b79f13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:39.743190 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a517bbe3-eda4-48b0-92f7-416850b79f13 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:39.743578 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a517bbe3-eda4-48b0-92f7-416850b79f13 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:39.743985 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a517bbe3-eda4-48b0-92f7-416850b79f13 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:39.745316 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a517bbe3-eda4-48b0-92f7-416850b79f13 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:39.747001 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a517bbe3-eda4-48b0-92f7-416850b79f13 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:39.791975 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-a517bbe3-eda4-48b0-92f7-416850b79f13 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 26 14:55:39.791975 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:55:39.791975 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 14:55:39.791975 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 14:55:39.791975 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 14:55:39.791975 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 14:55:39.791975 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 14:55:39.791975 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 14:55:39.791975 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 26 14:55:39.791975 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 26 14:55:39.791975 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 14:55:39.791975 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 26 14:55:39.791975 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:55:39.791975 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:55:39.791975 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:55:39.791975 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:55:39.793012 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:55:39.793012 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:55:39.793012 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:55:39.793012 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:55:39.793012 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:55:39.793012 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:55:39.793012 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:55:39.793012 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 14:55:39.793012 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 14:55:39.793012 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 14:55:39.793012 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 14:55:39.793012 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:55:39.793012 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:55:39.793012 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:55:39.793012 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:55:39.793012 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:55:39.793900 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:55:39.793900 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:55:39.793900 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 26 14:55:39.793900 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:55:39.793900 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 26 14:55:39.793900 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:55:39.793900 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:55:39.793900 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:55:39.793900 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 26 14:55:39.793900 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:55:39.793900 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 26 14:55:39.793900 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 14:55:39.793900 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 14:55:39.793900 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 26 14:55:39.793900 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 26 14:55:39.794704 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 376/762] 10.222.0.46 () {56 vars in 1129 bytes} [Thu May 26 14:55:39 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:55:39.802378 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a517bbe3-eda4-48b0-92f7-416850b79f13 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:39.879944 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c4f57e-47a0-4c31-839a-d2c2c044a7de None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:39.917859 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:39.930420 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e43ef04-99b9-4d94-9dae-518c78d7fc08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:39.932788 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e43ef04-99b9-4d94-9dae-518c78d7fc08 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:39.933084 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e43ef04-99b9-4d94-9dae-518c78d7fc08 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:39.933601 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e43ef04-99b9-4d94-9dae-518c78d7fc08 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:39.956915 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e43ef04-99b9-4d94-9dae-518c78d7fc08 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:39.961533 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e43ef04-99b9-4d94-9dae-518c78d7fc08 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:40.000516 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-5e43ef04-99b9-4d94-9dae-518c78d7fc08 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 26 14:55:40.000516 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:55:40.000516 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:55:40.000516 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:55:40.000516 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:55:40.000516 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:55:40.000516 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:55:40.000516 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:55:40.000516 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:55:40.000516 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:55:40.000516 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:55:40.000516 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:55:40.000516 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 14:55:40.000516 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 14:55:40.000516 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 14:55:40.000516 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 14:55:40.002275 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:55:40.002275 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:55:40.002275 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 14:55:40.002275 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 14:55:40.002275 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 14:55:40.002275 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 14:55:40.002275 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:55:40.002275 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:55:40.002275 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:55:40.002275 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:55:40.002275 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:55:40.002275 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 26 14:55:40.002275 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:55:40.002275 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 26 14:55:40.002275 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:55:40.002275 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:55:40.002275 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:55:40.002275 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 26 14:55:40.003660 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:55:40.003660 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 26 14:55:40.003660 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 14:55:40.003660 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 14:55:40.003660 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 14:55:40.003660 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 14:55:40.003660 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 14:55:40.003660 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 14:55:40.003660 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 26 14:55:40.003660 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 26 14:55:40.004459 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 387/763] 10.222.0.46 () {56 vars in 1093 bytes} [Thu May 26 14:55:39 2022] GET /identity/v3/users/nova => generated 82 bytes in 306 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:55:40.011919 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e43ef04-99b9-4d94-9dae-518c78d7fc08 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:40.099277 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a517bbe3-eda4-48b0-92f7-416850b79f13 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:40.166818 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c51d427-46cf-4ae1-9702-8b3beb751333 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:40.167393 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c51d427-46cf-4ae1-9702-8b3beb751333 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:40.167681 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c51d427-46cf-4ae1-9702-8b3beb751333 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:40.167962 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c51d427-46cf-4ae1-9702-8b3beb751333 None admin] PATH_INFO: `/v3/services` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:40.168685 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c51d427-46cf-4ae1-9702-8b3beb751333 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:55:40.169009 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c51d427-46cf-4ae1-9702-8b3beb751333 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:40.171103 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c51d427-46cf-4ae1-9702-8b3beb751333 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:40.185476 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 377/764] 10.222.0.46 () {56 vars in 1124 bytes} [Thu May 26 14:55:39 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 378 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:55:40.185476 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70ce169-0601-4e5f-8edf-be05cb8aa304 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:40.185476 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70ce169-0601-4e5f-8edf-be05cb8aa304 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:40.185476 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70ce169-0601-4e5f-8edf-be05cb8aa304 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:40.185476 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 378/765] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 14:55:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 14:55:40.187120 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e43ef04-99b9-4d94-9dae-518c78d7fc08 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:40.189928 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70ce169-0601-4e5f-8edf-be05cb8aa304 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:40.230916 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3f74ec-a43a-4dc0-90c8-0233fe84c18a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:40.231615 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3f74ec-a43a-4dc0-90c8-0233fe84c18a None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:40.231899 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3f74ec-a43a-4dc0-90c8-0233fe84c18a None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:40.232182 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3f74ec-a43a-4dc0-90c8-0233fe84c18a None admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:40.233161 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3f74ec-a43a-4dc0-90c8-0233fe84c18a None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:55:40.233499 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3f74ec-a43a-4dc0-90c8-0233fe84c18a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:40.234993 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3f74ec-a43a-4dc0-90c8-0233fe84c18a None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:40.261024 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 388/766] 10.222.0.46 () {56 vars in 1133 bytes} [Thu May 26 14:55:40 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:55:40.261024 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3f1dd6-8bf8-415b-927b-2c18543f220e None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:40.261024 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3f1dd6-8bf8-415b-927b-2c18543f220e None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:40.261024 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3f1dd6-8bf8-415b-927b-2c18543f220e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:40.280027 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-5c3f1dd6-8bf8-415b-927b-2c18543f220e None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:55:40.280849 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:40.281264 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-5c3f1dd6-8bf8-415b-927b-2c18543f220e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:55:40.336961 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70ce169-0601-4e5f-8edf-be05cb8aa304 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:40.339475 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:40.373479 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 389/767] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 14:55:40 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 14:55:40.373479 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3f1dd6-8bf8-415b-927b-2c18543f220e None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:40.398979 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300118c4-a1c3-4cde-847b-768f3cf4dd58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:40.400216 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300118c4-a1c3-4cde-847b-768f3cf4dd58 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:40.401322 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300118c4-a1c3-4cde-847b-768f3cf4dd58 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:40.401822 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300118c4-a1c3-4cde-847b-768f3cf4dd58 None admin] PATH_INFO: `/v3/services` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:40.402797 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300118c4-a1c3-4cde-847b-768f3cf4dd58 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:55:40.403378 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300118c4-a1c3-4cde-847b-768f3cf4dd58 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:40.405323 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300118c4-a1c3-4cde-847b-768f3cf4dd58 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:40.423944 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 379/768] 10.222.0.46 () {56 vars in 1124 bytes} [Thu May 26 14:55:40 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:55:40.425635 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300118c4-a1c3-4cde-847b-768f3cf4dd58 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:40.485793 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3f1dd6-8bf8-415b-927b-2c18543f220e None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:40.529171 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6edacc-7941-4cfe-85c4-dc18b35c6b4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:40.530359 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6edacc-7941-4cfe-85c4-dc18b35c6b4b None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:40.530762 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6edacc-7941-4cfe-85c4-dc18b35c6b4b None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:40.531248 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6edacc-7941-4cfe-85c4-dc18b35c6b4b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:40.546193 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6edacc-7941-4cfe-85c4-dc18b35c6b4b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:40.547781 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6edacc-7941-4cfe-85c4-dc18b35c6b4b None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:40.553289 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300118c4-a1c3-4cde-847b-768f3cf4dd58 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:40.567304 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-6b6edacc-7941-4cfe-85c4-dc18b35c6b4b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 14:55:40.567304 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:55:40.567304 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:55:40.567304 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:55:40.567304 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:55:40.567304 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:55:40.567304 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:55:40.567304 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:55:40.567304 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:55:40.567304 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:55:40.567304 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:55:40.567304 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:55:40.567304 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 14:55:40.567304 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 14:55:40.567304 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 14:55:40.568262 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 14:55:40.568262 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:55:40.568262 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:55:40.568262 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:55:40.568262 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:55:40.568262 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:55:40.568262 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:55:40.568262 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:55:40.568262 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 26 14:55:40.568262 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:55:40.568262 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 26 14:55:40.568262 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:55:40.568262 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:55:40.568262 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:55:40.568262 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 26 14:55:40.568262 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:55:40.568262 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 26 14:55:40.569137 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 14:55:40.569137 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 14:55:40.569137 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 14:55:40.569137 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 26 14:55:40.573810 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 390/769] 10.222.0.46 () {56 vars in 1108 bytes} [Thu May 26 14:55:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:55:40.574805 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6edacc-7941-4cfe-85c4-dc18b35c6b4b None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:40.577480 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:40.597105 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae759a7c-1bfa-47ab-a8d2-bc8c77476b2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:40.598400 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae759a7c-1bfa-47ab-a8d2-bc8c77476b2f None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:40.599016 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae759a7c-1bfa-47ab-a8d2-bc8c77476b2f None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:40.599490 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae759a7c-1bfa-47ab-a8d2-bc8c77476b2f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:40.614440 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:40.623818 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae759a7c-1bfa-47ab-a8d2-bc8c77476b2f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 14:55:40.633351 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-ae759a7c-1bfa-47ab-a8d2-bc8c77476b2f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 26 14:55:40.633351 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:55:40.633351 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:55:40.633351 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:55:40.633351 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:55:40.633351 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:55:40.633351 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:55:40.633351 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:55:40.633351 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:55:40.633351 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:55:40.633351 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:55:40.633351 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:55:40.633351 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 14:55:40.633351 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 14:55:40.633351 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 14:55:40.633351 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 14:55:40.634389 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 14:55:40.634389 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 14:55:40.634389 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 14:55:40.634389 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 14:55:40.634389 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:55:40.634389 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:55:40.634389 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:55:40.634389 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:55:40.634389 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:55:40.634389 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:55:40.634389 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:55:40.634389 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 26 14:55:40.634389 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:55:40.634389 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 26 14:55:40.634389 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:55:40.634389 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:55:40.635790 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:55:40.635790 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 26 14:55:40.635790 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:55:40.635790 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 26 14:55:40.635790 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 14:55:40.635790 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 14:55:40.635790 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 14:55:40.635790 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 14:55:40.635790 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 14:55:40.635790 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 14:55:40.635790 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 26 14:55:40.635790 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 26 14:55:40.643812 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 380/770] 10.222.0.46 () {56 vars in 1102 bytes} [Thu May 26 14:55:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:55:40.645156 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae759a7c-1bfa-47ab-a8d2-bc8c77476b2f None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:40.713636 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6edacc-7941-4cfe-85c4-dc18b35c6b4b None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:40.757386 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae759a7c-1bfa-47ab-a8d2-bc8c77476b2f None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:40.766666 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0964fe-b31e-4196-b135-a9e46adb00bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:40.767242 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0964fe-b31e-4196-b135-a9e46adb00bd None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:40.767751 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0964fe-b31e-4196-b135-a9e46adb00bd None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:40.768214 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0964fe-b31e-4196-b135-a9e46adb00bd None admin] PATH_INFO: `/v3/endpoints` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:40.769331 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0964fe-b31e-4196-b135-a9e46adb00bd None admin] RBAC: Adding query filter params (interface=public, service_id=2cea0d3a65ad4315b73a919a0f59275f, region_id=RegionOne) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:55:40.769794 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0964fe-b31e-4196-b135-a9e46adb00bd None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:40.771909 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0964fe-b31e-4196-b135-a9e46adb00bd None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:40.785502 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 391/771] 10.222.0.46 () {56 vars in 1251 bytes} [Thu May 26 14:55:40 2022] GET /identity/v3/endpoints?service_id=2cea0d3a65ad4315b73a919a0f59275f&interface=public®ion_id=RegionOne => generated 194 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:55:40.785502 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0964fe-b31e-4196-b135-a9e46adb00bd None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:40.803038 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fdce1d-207b-4422-8bb4-cd67ed0b06c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:40.804051 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fdce1d-207b-4422-8bb4-cd67ed0b06c0 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:40.804535 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fdce1d-207b-4422-8bb4-cd67ed0b06c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:40.804953 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fdce1d-207b-4422-8bb4-cd67ed0b06c0 None admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:40.807096 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86fdce1d-207b-4422-8bb4-cd67ed0b06c0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:55:40.807805 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86fdce1d-207b-4422-8bb4-cd67ed0b06c0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:40.810091 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86fdce1d-207b-4422-8bb4-cd67ed0b06c0 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:40.824590 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 381/772] 10.222.0.46 () {56 vars in 1109 bytes} [Thu May 26 14:55:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:55:40.825763 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fdce1d-207b-4422-8bb4-cd67ed0b06c0 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:40.870595 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:40.936358 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:40.946220 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0964fe-b31e-4196-b135-a9e46adb00bd None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:40.948445 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fdce1d-207b-4422-8bb4-cd67ed0b06c0 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:40.998067 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86b73fe-0313-4273-9e3d-b64639f4b6ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:40.998740 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86b73fe-0313-4273-9e3d-b64639f4b6ff None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:40.999040 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86b73fe-0313-4273-9e3d-b64639f4b6ff None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:40.999290 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86b73fe-0313-4273-9e3d-b64639f4b6ff None admin] PATH_INFO: `/v3/services` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:40.999675 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ee0e1a-6d98-4009-9003-95ea9cf0048a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:40.999980 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86b73fe-0313-4273-9e3d-b64639f4b6ff None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:41.000352 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ee0e1a-6d98-4009-9003-95ea9cf0048a None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:41.000637 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ee0e1a-6d98-4009-9003-95ea9cf0048a None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:41.000926 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ee0e1a-6d98-4009-9003-95ea9cf0048a None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:41.001360 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86b73fe-0313-4273-9e3d-b64639f4b6ff None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:41.001719 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ee0e1a-6d98-4009-9003-95ea9cf0048a None admin] RBAC: Adding query filter params (name=service) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:55:41.002030 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ee0e1a-6d98-4009-9003-95ea9cf0048a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:41.003682 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ee0e1a-6d98-4009-9003-95ea9cf0048a None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:41.018594 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 382/773] 10.222.0.46 () {56 vars in 1087 bytes} [Thu May 26 14:55:40 2022] GET /identity/v3/services => generated 772 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:55:41.018594 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86b73fe-0313-4273-9e3d-b64639f4b6ff None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:41.023746 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 392/774] 10.222.0.46 () {56 vars in 1103 bytes} [Thu May 26 14:55:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:55:41.025846 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455b59ce-bff8-4e99-824b-9909ae465bf2 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:41.026157 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455b59ce-bff8-4e99-824b-9909ae465bf2 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:41.026619 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455b59ce-bff8-4e99-824b-9909ae465bf2 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:41.028952 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 393/775] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 14:55:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 14:55:41.030244 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455b59ce-bff8-4e99-824b-9909ae465bf2 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:41.161567 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86b73fe-0313-4273-9e3d-b64639f4b6ff None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:41.163895 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:41.182886 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455b59ce-bff8-4e99-824b-9909ae465bf2 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:41.213465 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34bd0b36-524e-449f-9269-c06c34b68c78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:41.214507 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bd0b36-524e-449f-9269-c06c34b68c78 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:41.214789 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bd0b36-524e-449f-9269-c06c34b68c78 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:41.215064 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34bd0b36-524e-449f-9269-c06c34b68c78 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:41.227839 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34bd0b36-524e-449f-9269-c06c34b68c78 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:41.229852 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34bd0b36-524e-449f-9269-c06c34b68c78 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:41.239240 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329ed6ef-3fad-4cf5-b39e-ac6306b2f28a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:41.239240 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329ed6ef-3fad-4cf5-b39e-ac6306b2f28a None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:41.240738 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329ed6ef-3fad-4cf5-b39e-ac6306b2f28a None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:41.240738 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329ed6ef-3fad-4cf5-b39e-ac6306b2f28a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:41.255622 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-34bd0b36-524e-449f-9269-c06c34b68c78 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 14:55:41.255622 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:55:41.255622 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:55:41.255622 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:55:41.255622 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:55:41.255622 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:55:41.255622 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:55:41.255622 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:55:41.255622 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:55:41.255622 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:55:41.255622 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:55:41.255622 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:55:41.255622 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 14:55:41.255622 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 14:55:41.255622 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 14:55:41.256616 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 14:55:41.256616 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:55:41.256616 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:55:41.256616 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:55:41.256616 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:55:41.256616 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:55:41.256616 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:55:41.256616 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:55:41.256616 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 26 14:55:41.256616 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:55:41.256616 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 26 14:55:41.256616 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:55:41.256616 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:55:41.256616 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:55:41.256616 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 26 14:55:41.256616 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:55:41.256616 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 26 14:55:41.257613 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 14:55:41.257613 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 14:55:41.257613 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 14:55:41.257613 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 14:55:41.257613 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 14:55:41.257613 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 14:55:41.257613 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 14:55:41.257613 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 26 14:55:41.258058 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329ed6ef-3fad-4cf5-b39e-ac6306b2f28a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:41.258137 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 383/776] 10.222.0.46 () {56 vars in 1111 bytes} [Thu May 26 14:55:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:55:41.258209 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329ed6ef-3fad-4cf5-b39e-ac6306b2f28a None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:41.259718 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5475ad8-8808-434d-b584-d89d802f45b2 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:41.259718 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5475ad8-8808-434d-b584-d89d802f45b2 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:41.259718 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5475ad8-8808-434d-b584-d89d802f45b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:41.280400 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-c5475ad8-8808-434d-b584-d89d802f45b2 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:55:41.281324 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-329ed6ef-3fad-4cf5-b39e-ac6306b2f28a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 14:55:41.281324 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:55:41.281324 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:55:41.281324 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:55:41.281324 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:55:41.281324 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:55:41.281324 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:55:41.281324 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:55:41.281324 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:55:41.281324 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:55:41.281324 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:55:41.281324 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:55:41.281324 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 14:55:41.281324 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 14:55:41.281324 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 14:55:41.282590 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 14:55:41.282590 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:55:41.282590 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:55:41.282590 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:55:41.282590 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:55:41.282590 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:55:41.282590 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:55:41.282590 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:55:41.282590 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 26 14:55:41.282590 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:55:41.282590 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 26 14:55:41.282590 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:55:41.282590 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:55:41.282590 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:55:41.282590 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 26 14:55:41.282590 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:55:41.282590 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 26 14:55:41.284631 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 14:55:41.284631 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 14:55:41.284631 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 14:55:41.284631 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 26 14:55:41.285302 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-c5475ad8-8808-434d-b584-d89d802f45b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:55:41.285400 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 394/777] 10.222.0.46 () {56 vars in 1108 bytes} [Thu May 26 14:55:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:55:41.285400 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae8d28a-a9ae-45c1-9f20-f2bad8b7785e None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:41.285400 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae8d28a-a9ae-45c1-9f20-f2bad8b7785e None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:41.285400 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae8d28a-a9ae-45c1-9f20-f2bad8b7785e None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:41.286284 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 395/778] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 14:55:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 14:55:41.287002 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae8d28a-a9ae-45c1-9f20-f2bad8b7785e None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:41.303645 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:41.375989 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 384/779] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 14:55:41 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 14:55:41.377300 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5475ad8-8808-434d-b584-d89d802f45b2 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:41.444730 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae8d28a-a9ae-45c1-9f20-f2bad8b7785e None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:41.491263 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d6ea90-dd87-440d-9a9d-76ce87144548 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:41.491935 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d6ea90-dd87-440d-9a9d-76ce87144548 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:41.492288 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d6ea90-dd87-440d-9a9d-76ce87144548 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:41.492540 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d6ea90-dd87-440d-9a9d-76ce87144548 None admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:41.493344 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d6ea90-dd87-440d-9a9d-76ce87144548 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:55:41.493713 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d6ea90-dd87-440d-9a9d-76ce87144548 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:41.495196 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d6ea90-dd87-440d-9a9d-76ce87144548 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:41.503818 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:41.518373 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 396/780] 10.222.0.46 () {56 vars in 1148 bytes} [Thu May 26 14:55:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:55:41.518373 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e77008c-ea7e-477e-b43c-b49452d847e0 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:41.518373 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e77008c-ea7e-477e-b43c-b49452d847e0 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:41.518373 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e77008c-ea7e-477e-b43c-b49452d847e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:41.523979 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5475ad8-8808-434d-b584-d89d802f45b2 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:41.536432 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-0e77008c-ea7e-477e-b43c-b49452d847e0 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:55:41.537704 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-0e77008c-ea7e-477e-b43c-b49452d847e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:55:41.582630 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861db4c3-3486-4dbd-bf2b-6f3482ab71ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:41.583832 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861db4c3-3486-4dbd-bf2b-6f3482ab71ea None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:41.583929 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861db4c3-3486-4dbd-bf2b-6f3482ab71ea None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:41.584322 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-861db4c3-3486-4dbd-bf2b-6f3482ab71ea None admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:41.585064 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861db4c3-3486-4dbd-bf2b-6f3482ab71ea None admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:55:41.585716 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861db4c3-3486-4dbd-bf2b-6f3482ab71ea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:41.592900 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-861db4c3-3486-4dbd-bf2b-6f3482ab71ea None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:41.609617 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 385/781] 10.222.0.46 () {56 vars in 1109 bytes} [Thu May 26 14:55:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:55:41.610534 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861db4c3-3486-4dbd-bf2b-6f3482ab71ea None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:41.633055 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:41.638502 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 397/782] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 14:55:41 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 14:55:41.639163 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e77008c-ea7e-477e-b43c-b49452d847e0 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:41.751821 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-861db4c3-3486-4dbd-bf2b-6f3482ab71ea None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:41.755938 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e77008c-ea7e-477e-b43c-b49452d847e0 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:41.799592 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e920b1-802e-4ac2-a9ee-1d6cc321be25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:41.800502 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e920b1-802e-4ac2-a9ee-1d6cc321be25 None admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:41.800787 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e920b1-802e-4ac2-a9ee-1d6cc321be25 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:41.801067 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e920b1-802e-4ac2-a9ee-1d6cc321be25 None admin] PATH_INFO: `/v3/projects/a17fc8227d4c4da6b2847b350e912ee3/users/d6639353c70449d1b3d10cd6487c3b00/roles/a2e551773cbd4df084020676865b8222` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:41.803247 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6edaaa-50d8-4f43-a7cc-84a117136044 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:41.803408 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e920b1-802e-4ac2-a9ee-1d6cc321be25 None admin] RBAC: Authorizing `identity:create_grant(project_id=a17fc8227d4c4da6b2847b350e912ee3, user_id=d6639353c70449d1b3d10cd6487c3b00, role_id=a2e551773cbd4df084020676865b8222)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:41.804471 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6edaaa-50d8-4f43-a7cc-84a117136044 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:41.804751 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e920b1-802e-4ac2-a9ee-1d6cc321be25 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:41.805070 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6edaaa-50d8-4f43-a7cc-84a117136044 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:41.805614 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6edaaa-50d8-4f43-a7cc-84a117136044 None admin] PATH_INFO: `/v3/services/network` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:41.806445 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6edaaa-50d8-4f43-a7cc-84a117136044 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:41.807987 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6edaaa-50d8-4f43-a7cc-84a117136044 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:41.821066 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-ae6edaaa-50d8-4f43-a7cc-84a117136044 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 26 14:55:41.821066 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:55:41.821066 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 14:55:41.821066 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 14:55:41.821066 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 14:55:41.821066 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 14:55:41.821066 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 14:55:41.821066 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 14:55:41.821066 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 26 14:55:41.821066 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 26 14:55:41.821066 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 14:55:41.821066 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 26 14:55:41.821066 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:55:41.821066 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:55:41.821066 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:55:41.821066 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:55:41.821066 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:55:41.822845 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:55:41.822845 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:55:41.822845 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:55:41.822845 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:55:41.822845 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:55:41.822845 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:55:41.822845 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 14:55:41.822845 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 14:55:41.822845 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 14:55:41.822845 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 14:55:41.822845 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:55:41.822845 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:55:41.822845 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:55:41.822845 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:55:41.822845 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:55:41.822845 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:55:41.824541 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:55:41.824541 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 26 14:55:41.824541 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:55:41.824541 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 26 14:55:41.824541 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:55:41.824541 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:55:41.824541 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:55:41.824541 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 26 14:55:41.824541 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:55:41.824541 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 26 14:55:41.824541 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 14:55:41.824541 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 14:55:41.824541 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 26 14:55:41.824541 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 26 14:55:41.827864 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 386/784] 10.222.0.46 () {56 vars in 1111 bytes} [Thu May 26 14:55:41 2022] GET /identity/v3/services/network => generated 88 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:55:41.828854 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6edaaa-50d8-4f43-a7cc-84a117136044 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:41.829555 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 398/784] 10.222.0.46 () {58 vars in 1439 bytes} [Thu May 26 14:55:41 2022] PUT /identity/v3/projects/a17fc8227d4c4da6b2847b350e912ee3/users/d6639353c70449d1b3d10cd6487c3b00/roles/a2e551773cbd4df084020676865b8222 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 14:55:41.831231 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e920b1-802e-4ac2-a9ee-1d6cc321be25 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:41.874107 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:41.924378 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:41.984567 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e920b1-802e-4ac2-a9ee-1d6cc321be25 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:41.995814 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6edaaa-50d8-4f43-a7cc-84a117136044 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:42.040833 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92ee3eb-269d-47c6-966f-daae610a0644 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:42.042116 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92ee3eb-269d-47c6-966f-daae610a0644 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:42.042616 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92ee3eb-269d-47c6-966f-daae610a0644 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:42.043096 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92ee3eb-269d-47c6-966f-daae610a0644 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:42.057162 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09861858-699e-46a8-9495-3073dfd549e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:42.058342 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09861858-699e-46a8-9495-3073dfd549e5 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:42.058827 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09861858-699e-46a8-9495-3073dfd549e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:42.059290 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09861858-699e-46a8-9495-3073dfd549e5 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:42.060038 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92ee3eb-269d-47c6-966f-daae610a0644 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 14:55:42.069063 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09861858-699e-46a8-9495-3073dfd549e5 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:42.069684 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-f92ee3eb-269d-47c6-966f-daae610a0644 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 14:55:42.069684 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:55:42.069684 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:55:42.069684 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:55:42.069684 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:55:42.069684 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:55:42.069684 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:55:42.069684 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:55:42.069684 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:55:42.069684 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:55:42.069684 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:55:42.069684 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:55:42.069684 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 14:55:42.069684 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 14:55:42.069684 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 14:55:42.069684 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 14:55:42.070625 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 14:55:42.070625 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 14:55:42.070625 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 14:55:42.070625 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 14:55:42.070625 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:55:42.070625 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:55:42.070625 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:55:42.070625 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:55:42.070625 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:55:42.070625 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:55:42.070625 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:55:42.070625 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 26 14:55:42.070625 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:55:42.070625 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 26 14:55:42.070625 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:55:42.070625 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:55:42.071476 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:55:42.071476 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 26 14:55:42.071476 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:55:42.071476 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 26 14:55:42.071476 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 14:55:42.071476 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 14:55:42.071476 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 14:55:42.071476 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 14:55:42.071476 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 14:55:42.071476 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 14:55:42.071476 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 14:55:42.071476 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 26 14:55:42.071476 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 399/785] 10.222.0.46 () {56 vars in 1096 bytes} [Thu May 26 14:55:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:55:42.072742 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92ee3eb-269d-47c6-966f-daae610a0644 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:42.074278 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09861858-699e-46a8-9495-3073dfd549e5 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:42.096219 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-09861858-699e-46a8-9495-3073dfd549e5 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 26 14:55:42.096219 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:55:42.096219 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:55:42.096219 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:55:42.096219 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:55:42.096219 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:55:42.096219 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:55:42.096219 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:55:42.096219 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:55:42.096219 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:55:42.096219 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:55:42.096219 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:55:42.096219 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 14:55:42.096219 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 14:55:42.096219 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 14:55:42.096219 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 14:55:42.097260 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:55:42.097260 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:55:42.097260 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 14:55:42.097260 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 14:55:42.097260 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 14:55:42.097260 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 14:55:42.097260 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:55:42.097260 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:55:42.097260 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:55:42.097260 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:55:42.097260 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:55:42.097260 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 26 14:55:42.097260 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:55:42.097260 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 26 14:55:42.097260 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:55:42.097260 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:55:42.097260 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:55:42.097260 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 26 14:55:42.098338 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:55:42.098338 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 26 14:55:42.098338 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 14:55:42.098338 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 14:55:42.098338 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 14:55:42.098338 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 14:55:42.098338 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 14:55:42.098338 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 14:55:42.098338 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 26 14:55:42.098338 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 26 14:55:42.098338 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 387/786] 10.222.0.46 () {56 vars in 1117 bytes} [Thu May 26 14:55:41 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:55:42.098952 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09861858-699e-46a8-9495-3073dfd549e5 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:42.213928 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:42.241833 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09861858-699e-46a8-9495-3073dfd549e5 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:42.258996 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92ee3eb-269d-47c6-966f-daae610a0644 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:42.288484 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e635ed-8c30-4e10-9536-e25d54ebbe51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:42.289934 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e635ed-8c30-4e10-9536-e25d54ebbe51 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:42.290394 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e635ed-8c30-4e10-9536-e25d54ebbe51 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:42.290846 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e635ed-8c30-4e10-9536-e25d54ebbe51 None admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:42.291919 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e635ed-8c30-4e10-9536-e25d54ebbe51 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:55:42.292457 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e635ed-8c30-4e10-9536-e25d54ebbe51 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:42.294119 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e635ed-8c30-4e10-9536-e25d54ebbe51 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:42.295474 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:42.310694 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 388/787] 10.222.0.46 () {56 vars in 1099 bytes} [Thu May 26 14:55:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:55:42.312264 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e635ed-8c30-4e10-9536-e25d54ebbe51 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:42.326927 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07f76d3-826b-449e-97fb-d454b233c961 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:42.326927 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07f76d3-826b-449e-97fb-d454b233c961 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:42.326927 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07f76d3-826b-449e-97fb-d454b233c961 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:42.326927 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07f76d3-826b-449e-97fb-d454b233c961 None admin] PATH_INFO: `/v3/services` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:42.327396 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07f76d3-826b-449e-97fb-d454b233c961 None admin] RBAC: Adding query filter params (name=network) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:55:42.327396 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07f76d3-826b-449e-97fb-d454b233c961 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:42.327606 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07f76d3-826b-449e-97fb-d454b233c961 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:42.336302 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 400/788] 10.222.0.46 () {56 vars in 1112 bytes} [Thu May 26 14:55:42 2022] GET /identity/v3/services?name=network => generated 124 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:55:42.337892 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07f76d3-826b-449e-97fb-d454b233c961 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:42.429296 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e635ed-8c30-4e10-9536-e25d54ebbe51 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:42.467867 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b82c37-f9e7-493c-ac93-be7daac303d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:42.468556 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b82c37-f9e7-493c-ac93-be7daac303d6 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:42.468841 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b82c37-f9e7-493c-ac93-be7daac303d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:42.469114 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b82c37-f9e7-493c-ac93-be7daac303d6 None admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:42.469990 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b82c37-f9e7-493c-ac93-be7daac303d6 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:55:42.470299 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b82c37-f9e7-493c-ac93-be7daac303d6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:42.471875 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b82c37-f9e7-493c-ac93-be7daac303d6 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:42.481373 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:42.483667 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07f76d3-826b-449e-97fb-d454b233c961 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:42.499937 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 389/789] 10.222.0.46 () {56 vars in 1149 bytes} [Thu May 26 14:55:42 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:55:42.500908 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b82c37-f9e7-493c-ac93-be7daac303d6 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:42.525207 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33ae1a3-b203-452b-9715-086913fb6518 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:42.526561 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33ae1a3-b203-452b-9715-086913fb6518 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:42.527053 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33ae1a3-b203-452b-9715-086913fb6518 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:42.527546 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33ae1a3-b203-452b-9715-086913fb6518 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:42.540380 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33ae1a3-b203-452b-9715-086913fb6518 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:42.542525 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33ae1a3-b203-452b-9715-086913fb6518 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:42.562157 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-a33ae1a3-b203-452b-9715-086913fb6518 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 14:55:42.562157 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:55:42.562157 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:55:42.562157 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:55:42.562157 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:55:42.562157 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:55:42.562157 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:55:42.562157 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:55:42.562157 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:55:42.562157 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:55:42.562157 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:55:42.562157 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:55:42.562157 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 14:55:42.562157 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 14:55:42.562157 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 14:55:42.563155 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 14:55:42.563155 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:55:42.563155 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:55:42.563155 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:55:42.563155 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:55:42.563155 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:55:42.563155 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:55:42.563155 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:55:42.563155 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 26 14:55:42.563155 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:55:42.563155 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 26 14:55:42.563155 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:55:42.563155 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:55:42.563155 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:55:42.563155 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 26 14:55:42.563155 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:55:42.563155 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 26 14:55:42.564100 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 14:55:42.564100 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 14:55:42.564100 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 14:55:42.564100 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 26 14:55:42.568201 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 401/790] 10.222.0.46 () {56 vars in 1108 bytes} [Thu May 26 14:55:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:55:42.569200 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33ae1a3-b203-452b-9715-086913fb6518 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:42.614549 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:42.667497 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b82c37-f9e7-493c-ac93-be7daac303d6 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:42.705042 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33ae1a3-b203-452b-9715-086913fb6518 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:42.710768 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f0e02e-ad0c-48e1-b8c2-5b5011591b0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:42.711648 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f0e02e-ad0c-48e1-b8c2-5b5011591b0b None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:42.711936 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f0e02e-ad0c-48e1-b8c2-5b5011591b0b None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:42.712440 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f0e02e-ad0c-48e1-b8c2-5b5011591b0b None admin] PATH_INFO: `/v3/services` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:42.713552 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f0e02e-ad0c-48e1-b8c2-5b5011591b0b None admin] RBAC: Adding query filter params (type=network) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:55:42.713872 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f0e02e-ad0c-48e1-b8c2-5b5011591b0b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:42.715843 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f0e02e-ad0c-48e1-b8c2-5b5011591b0b None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:42.727494 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 390/791] 10.222.0.46 () {56 vars in 1112 bytes} [Thu May 26 14:55:42 2022] GET /identity/v3/services?type=network => generated 350 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:55:42.729485 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f0e02e-ad0c-48e1-b8c2-5b5011591b0b None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:42.744574 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5ba047-f730-4649-83db-946f817e040e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:42.745416 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5ba047-f730-4649-83db-946f817e040e None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:42.745747 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5ba047-f730-4649-83db-946f817e040e None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:42.746025 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5ba047-f730-4649-83db-946f817e040e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:42.758770 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5ba047-f730-4649-83db-946f817e040e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:42.760914 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5ba047-f730-4649-83db-946f817e040e None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:42.781136 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-fb5ba047-f730-4649-83db-946f817e040e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 14:55:42.781136 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:55:42.781136 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:55:42.781136 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:55:42.781136 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:55:42.781136 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:55:42.781136 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:55:42.781136 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:55:42.781136 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:55:42.781136 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:55:42.781136 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:55:42.781136 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:55:42.781136 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 14:55:42.781136 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 14:55:42.781136 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 14:55:42.782247 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 14:55:42.782247 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:55:42.782247 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:55:42.782247 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:55:42.782247 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:55:42.782247 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:55:42.782247 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:55:42.782247 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:55:42.782247 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 26 14:55:42.782247 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:55:42.782247 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 26 14:55:42.782247 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:55:42.782247 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:55:42.782247 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:55:42.782247 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 26 14:55:42.782247 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:55:42.782247 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 26 14:55:42.783131 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 14:55:42.783131 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 14:55:42.783131 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 14:55:42.783131 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 26 14:55:42.784116 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 402/792] 10.222.0.46 () {56 vars in 1108 bytes} [Thu May 26 14:55:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:55:42.784255 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5ba047-f730-4649-83db-946f817e040e None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:42.804274 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:42.863968 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f0e02e-ad0c-48e1-b8c2-5b5011591b0b None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:42.905235 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a081166-e10d-4b25-9520-b394ae952014 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:42.906302 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a081166-e10d-4b25-9520-b394ae952014 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:42.906509 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a081166-e10d-4b25-9520-b394ae952014 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:42.906931 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a081166-e10d-4b25-9520-b394ae952014 None admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:42.907893 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a081166-e10d-4b25-9520-b394ae952014 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:55:42.908314 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a081166-e10d-4b25-9520-b394ae952014 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:42.910082 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a081166-e10d-4b25-9520-b394ae952014 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:42.920934 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:42.926755 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 391/793] 10.222.0.46 () {56 vars in 1109 bytes} [Thu May 26 14:55:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:55:42.928199 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a081166-e10d-4b25-9520-b394ae952014 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:42.950471 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5ba047-f730-4649-83db-946f817e040e None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:43.010920 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a59ef9a-df95-4ff9-9232-0ce10c9c3721 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:43.011962 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a59ef9a-df95-4ff9-9232-0ce10c9c3721 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:43.012275 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a59ef9a-df95-4ff9-9232-0ce10c9c3721 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:43.012589 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a59ef9a-df95-4ff9-9232-0ce10c9c3721 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:43.013584 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a59ef9a-df95-4ff9-9232-0ce10c9c3721 None admin] RBAC: Adding query filter params (interface=public, service_id=ae1a771744ae468d8dff495128050e9c, region_id=RegionOne) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:55:43.014153 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a59ef9a-df95-4ff9-9232-0ce10c9c3721 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:43.016649 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a59ef9a-df95-4ff9-9232-0ce10c9c3721 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:43.031506 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 403/794] 10.222.0.46 () {56 vars in 1251 bytes} [Thu May 26 14:55:42 2022] GET /identity/v3/endpoints?service_id=ae1a771744ae468d8dff495128050e9c&interface=public®ion_id=RegionOne => generated 194 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:55:43.032709 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a59ef9a-df95-4ff9-9232-0ce10c9c3721 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:43.102255 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a081166-e10d-4b25-9520-b394ae952014 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:43.130815 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:43.151744 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738cbe62-d29e-4ad4-a6f9-d15716517983 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:43.152860 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738cbe62-d29e-4ad4-a6f9-d15716517983 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:43.153379 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738cbe62-d29e-4ad4-a6f9-d15716517983 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:43.153950 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738cbe62-d29e-4ad4-a6f9-d15716517983 None admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:43.154975 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738cbe62-d29e-4ad4-a6f9-d15716517983 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:55:43.155522 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738cbe62-d29e-4ad4-a6f9-d15716517983 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:43.157674 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738cbe62-d29e-4ad4-a6f9-d15716517983 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:43.179499 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 392/795] 10.222.0.46 () {56 vars in 1109 bytes} [Thu May 26 14:55:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:55:43.181119 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738cbe62-d29e-4ad4-a6f9-d15716517983 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:43.336461 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738cbe62-d29e-4ad4-a6f9-d15716517983 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:43.354797 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a59ef9a-df95-4ff9-9232-0ce10c9c3721 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:43.407349 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28f8883-978f-4e6a-b99b-7406f5943805 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:43.408177 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28f8883-978f-4e6a-b99b-7406f5943805 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:43.408597 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28f8883-978f-4e6a-b99b-7406f5943805 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:43.408747 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28f8883-978f-4e6a-b99b-7406f5943805 None admin] PATH_INFO: `/v3/services` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:43.409812 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28f8883-978f-4e6a-b99b-7406f5943805 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:43.412088 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28f8883-978f-4e6a-b99b-7406f5943805 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:43.424101 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 393/796] 10.222.0.46 () {56 vars in 1087 bytes} [Thu May 26 14:55:43 2022] GET /identity/v3/services => generated 772 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:55:43.425177 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099d4ef4-c7e5-4ba7-930a-7b458375f492 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:43.425930 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28f8883-978f-4e6a-b99b-7406f5943805 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:43.426274 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099d4ef4-c7e5-4ba7-930a-7b458375f492 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:43.426586 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099d4ef4-c7e5-4ba7-930a-7b458375f492 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:43.427064 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099d4ef4-c7e5-4ba7-930a-7b458375f492 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:43.442927 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099d4ef4-c7e5-4ba7-930a-7b458375f492 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:43.446090 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-099d4ef4-c7e5-4ba7-930a-7b458375f492 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:43.475136 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-099d4ef4-c7e5-4ba7-930a-7b458375f492 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 26 14:55:43.475136 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:55:43.475136 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:55:43.475136 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:55:43.475136 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:55:43.475136 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:55:43.475136 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:55:43.475136 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:55:43.475136 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:55:43.475136 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:55:43.475136 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:55:43.475136 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:55:43.475136 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 14:55:43.475136 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 14:55:43.475136 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 14:55:43.475136 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 14:55:43.476454 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:55:43.476454 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:55:43.476454 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 14:55:43.476454 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 14:55:43.476454 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 14:55:43.476454 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 14:55:43.476454 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:55:43.476454 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:55:43.476454 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:55:43.476454 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:55:43.476454 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:55:43.476454 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 26 14:55:43.476454 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:55:43.476454 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 26 14:55:43.476454 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:55:43.476454 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:55:43.476454 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:55:43.476454 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 26 14:55:43.477587 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:55:43.477587 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 26 14:55:43.477587 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 14:55:43.477587 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 14:55:43.477587 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 14:55:43.477587 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 14:55:43.477587 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 14:55:43.477587 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 14:55:43.477587 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 26 14:55:43.477587 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 26 14:55:43.482185 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 404/797] 10.222.0.46 () {56 vars in 1096 bytes} [Thu May 26 14:55:43 2022] GET /identity/v3/users/swift => generated 83 bytes in 450 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:55:43.485943 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25041632-7e70-4e8c-bbf6-4f038b9a5e21 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:43.486967 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25041632-7e70-4e8c-bbf6-4f038b9a5e21 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:43.488208 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25041632-7e70-4e8c-bbf6-4f038b9a5e21 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:43.491784 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 405/798] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 14:55:43 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 14:55:43.496055 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25041632-7e70-4e8c-bbf6-4f038b9a5e21 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:43.515103 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:43.532653 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:43.617243 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28f8883-978f-4e6a-b99b-7406f5943805 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:43.677501 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f211324d-ebfa-4700-8711-34d30acf22df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:43.678250 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f211324d-ebfa-4700-8711-34d30acf22df None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:43.678539 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f211324d-ebfa-4700-8711-34d30acf22df None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:43.678817 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f211324d-ebfa-4700-8711-34d30acf22df None admin] PATH_INFO: `/v3/projects/service` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:43.680096 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25041632-7e70-4e8c-bbf6-4f038b9a5e21 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:43.689614 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f211324d-ebfa-4700-8711-34d30acf22df None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:43.691097 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f211324d-ebfa-4700-8711-34d30acf22df None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:43.713958 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-f211324d-ebfa-4700-8711-34d30acf22df None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 14:55:43.713958 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:55:43.713958 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:55:43.713958 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:55:43.713958 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:55:43.713958 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:55:43.713958 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:55:43.713958 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:55:43.713958 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:55:43.713958 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:55:43.713958 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:55:43.713958 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:55:43.713958 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 14:55:43.713958 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 14:55:43.713958 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 14:55:43.714896 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 14:55:43.714896 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:55:43.714896 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:55:43.714896 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:55:43.714896 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:55:43.714896 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:55:43.714896 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:55:43.714896 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:55:43.714896 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 26 14:55:43.714896 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:55:43.714896 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 26 14:55:43.714896 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:55:43.714896 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:55:43.714896 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:55:43.714896 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 26 14:55:43.714896 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:55:43.714896 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 26 14:55:43.715790 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 14:55:43.715790 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 14:55:43.715790 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 14:55:43.715790 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 14:55:43.715790 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 14:55:43.715790 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 14:55:43.715790 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 14:55:43.715790 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 26 14:55:43.715790 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 394/799] 10.222.0.46 () {56 vars in 1111 bytes} [Thu May 26 14:55:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:55:43.718080 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebceaf3f-9058-40ec-81a2-d5851a81183b None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:43.718080 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebceaf3f-9058-40ec-81a2-d5851a81183b None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:43.718299 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebceaf3f-9058-40ec-81a2-d5851a81183b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:43.744010 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-ebceaf3f-9058-40ec-81a2-d5851a81183b None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:55:43.745256 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-ebceaf3f-9058-40ec-81a2-d5851a81183b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:55:43.749677 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7f002b-babf-4bd1-a5d8-c6f6265e337a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:43.750045 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7f002b-babf-4bd1-a5d8-c6f6265e337a None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:43.750415 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7f002b-babf-4bd1-a5d8-c6f6265e337a None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:43.750860 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7f002b-babf-4bd1-a5d8-c6f6265e337a None admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:43.752203 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7f002b-babf-4bd1-a5d8-c6f6265e337a None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:55:43.752665 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7f002b-babf-4bd1-a5d8-c6f6265e337a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:43.755217 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7f002b-babf-4bd1-a5d8-c6f6265e337a None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:43.792030 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 406/800] 10.222.0.46 () {56 vars in 1135 bytes} [Thu May 26 14:55:43 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:55:43.792030 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7f002b-babf-4bd1-a5d8-c6f6265e337a None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:43.848336 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 395/801] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 14:55:43 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 14:55:43.849723 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebceaf3f-9058-40ec-81a2-d5851a81183b None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:43.902650 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:43.934447 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:43.992301 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7f002b-babf-4bd1-a5d8-c6f6265e337a None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:43.997929 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebceaf3f-9058-40ec-81a2-d5851a81183b None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:44.041015 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d727580-4dee-4203-a24f-c82730e72476 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:44.042341 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d727580-4dee-4203-a24f-c82730e72476 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:44.043065 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d727580-4dee-4203-a24f-c82730e72476 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:44.043612 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d727580-4dee-4203-a24f-c82730e72476 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:44.054225 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849d645f-7545-4f84-a38c-2e6814465cba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:44.055182 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849d645f-7545-4f84-a38c-2e6814465cba None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:44.055526 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849d645f-7545-4f84-a38c-2e6814465cba None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:44.055855 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849d645f-7545-4f84-a38c-2e6814465cba None admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:44.058446 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849d645f-7545-4f84-a38c-2e6814465cba None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:55:44.058774 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849d645f-7545-4f84-a38c-2e6814465cba None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:44.060693 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d727580-4dee-4203-a24f-c82730e72476 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:44.061236 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849d645f-7545-4f84-a38c-2e6814465cba None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:44.062681 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d727580-4dee-4203-a24f-c82730e72476 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:44.088092 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 407/802] 10.222.0.46 () {56 vars in 1148 bytes} [Thu May 26 14:55:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:55:44.089600 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849d645f-7545-4f84-a38c-2e6814465cba None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:44.093576 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-4d727580-4dee-4203-a24f-c82730e72476 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 14:55:44.093576 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:55:44.093576 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:55:44.093576 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:55:44.093576 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:55:44.093576 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:55:44.093576 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:55:44.093576 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:55:44.093576 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:55:44.093576 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:55:44.093576 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:55:44.093576 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:55:44.093576 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 14:55:44.093576 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 14:55:44.093576 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 14:55:44.095027 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 14:55:44.095027 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:55:44.095027 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:55:44.095027 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:55:44.095027 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:55:44.095027 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:55:44.095027 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:55:44.095027 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:55:44.095027 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 26 14:55:44.095027 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:55:44.095027 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 26 14:55:44.095027 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:55:44.095027 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:55:44.095027 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:55:44.095027 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 26 14:55:44.095027 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:55:44.095027 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 26 14:55:44.096456 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 14:55:44.096456 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 14:55:44.096456 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 14:55:44.096456 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 26 14:55:44.099942 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 396/803] 10.222.0.46 () {56 vars in 1108 bytes} [Thu May 26 14:55:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:55:44.107089 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d727580-4dee-4203-a24f-c82730e72476 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:44.261429 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849d645f-7545-4f84-a38c-2e6814465cba None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:44.263748 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:44.286855 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d727580-4dee-4203-a24f-c82730e72476 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:44.309750 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:44.317642 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e3c9e1-0140-461d-afb0-31697f6b552c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:44.318509 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e3c9e1-0140-461d-afb0-31697f6b552c None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:44.318798 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e3c9e1-0140-461d-afb0-31697f6b552c None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:44.319079 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e3c9e1-0140-461d-afb0-31697f6b552c None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:44.319772 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e3c9e1-0140-461d-afb0-31697f6b552c None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:44.321523 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e3c9e1-0140-461d-afb0-31697f6b552c None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:44.337999 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-d4e3c9e1-0140-461d-afb0-31697f6b552c None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 26 14:55:44.337999 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:55:44.337999 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 14:55:44.337999 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 14:55:44.337999 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 14:55:44.337999 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 14:55:44.337999 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 14:55:44.337999 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 14:55:44.337999 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 26 14:55:44.337999 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 26 14:55:44.337999 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 14:55:44.337999 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 26 14:55:44.337999 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:55:44.337999 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:55:44.337999 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:55:44.337999 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:55:44.339611 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:55:44.339611 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:55:44.339611 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:55:44.339611 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:55:44.339611 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:55:44.339611 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:55:44.339611 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:55:44.339611 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 14:55:44.339611 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 14:55:44.339611 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 14:55:44.339611 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 14:55:44.339611 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:55:44.339611 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:55:44.339611 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:55:44.339611 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:55:44.339611 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:55:44.341052 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:55:44.341052 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:55:44.341052 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 26 14:55:44.341052 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:55:44.341052 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 26 14:55:44.341052 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:55:44.341052 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:55:44.341052 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:55:44.341052 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 26 14:55:44.341052 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:55:44.341052 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 26 14:55:44.341052 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 14:55:44.341052 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 14:55:44.341052 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 26 14:55:44.341052 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 26 14:55:44.344463 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 408/804] 10.222.0.46 () {56 vars in 1129 bytes} [Thu May 26 14:55:44 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:55:44.345439 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e3c9e1-0140-461d-afb0-31697f6b552c None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:44.357633 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cfad54-dbb1-4d5d-a496-03eaccfedc86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:44.359777 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cfad54-dbb1-4d5d-a496-03eaccfedc86 None admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:44.360553 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cfad54-dbb1-4d5d-a496-03eaccfedc86 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:44.361383 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cfad54-dbb1-4d5d-a496-03eaccfedc86 None admin] PATH_INFO: `/v3/projects/a17fc8227d4c4da6b2847b350e912ee3/users/8028c520d93449a6a087c281280ee13d/roles/eb76213820fa4c4e9b0a5d5ac2a08b41` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:44.391426 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0cfad54-dbb1-4d5d-a496-03eaccfedc86 None admin] RBAC: Authorizing `identity:create_grant(project_id=a17fc8227d4c4da6b2847b350e912ee3, user_id=8028c520d93449a6a087c281280ee13d, role_id=eb76213820fa4c4e9b0a5d5ac2a08b41)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:44.394356 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0cfad54-dbb1-4d5d-a496-03eaccfedc86 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:44.425800 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 397/805] 10.222.0.46 () {58 vars in 1439 bytes} [Thu May 26 14:55:44 2022] PUT /identity/v3/projects/a17fc8227d4c4da6b2847b350e912ee3/users/8028c520d93449a6a087c281280ee13d/roles/eb76213820fa4c4e9b0a5d5ac2a08b41 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 14:55:44.429740 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cfad54-dbb1-4d5d-a496-03eaccfedc86 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:44.571592 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e3c9e1-0140-461d-afb0-31697f6b552c None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:44.645178 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aced7797-12aa-4349-8167-737736a04bb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:44.646416 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aced7797-12aa-4349-8167-737736a04bb0 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:44.647365 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aced7797-12aa-4349-8167-737736a04bb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:44.647923 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aced7797-12aa-4349-8167-737736a04bb0 None admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:44.649740 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aced7797-12aa-4349-8167-737736a04bb0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:55:44.650834 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aced7797-12aa-4349-8167-737736a04bb0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:44.652846 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aced7797-12aa-4349-8167-737736a04bb0 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:44.654217 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cfad54-dbb1-4d5d-a496-03eaccfedc86 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:44.671023 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 409/806] 10.222.0.46 () {56 vars in 1109 bytes} [Thu May 26 14:55:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:55:44.673491 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d03a15c-1a14-4597-a264-e60a82acd23f None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:44.674232 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d03a15c-1a14-4597-a264-e60a82acd23f None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:44.674840 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d03a15c-1a14-4597-a264-e60a82acd23f None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:44.676770 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 410/807] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 14:55:44 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 14:55:44.679493 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d03a15c-1a14-4597-a264-e60a82acd23f None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:44.704131 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f2d188-3430-454f-ba56-df80e25c0117 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:44.705486 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f2d188-3430-454f-ba56-df80e25c0117 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:44.705962 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f2d188-3430-454f-ba56-df80e25c0117 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:44.706236 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f2d188-3430-454f-ba56-df80e25c0117 None admin] PATH_INFO: `/v3/services` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:44.706972 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f2d188-3430-454f-ba56-df80e25c0117 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:55:44.707276 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f2d188-3430-454f-ba56-df80e25c0117 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:44.708678 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f2d188-3430-454f-ba56-df80e25c0117 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:44.710875 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:44.716119 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 398/808] 10.222.0.46 () {56 vars in 1124 bytes} [Thu May 26 14:55:44 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:55:44.718119 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20977f2-8a0d-4209-9d07-d2df8ded1bac None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:44.718963 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20977f2-8a0d-4209-9d07-d2df8ded1bac None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:44.719402 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20977f2-8a0d-4209-9d07-d2df8ded1bac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:44.747793 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-b20977f2-8a0d-4209-9d07-d2df8ded1bac None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:55:44.749283 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-b20977f2-8a0d-4209-9d07-d2df8ded1bac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:55:44.761804 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:44.840737 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d03a15c-1a14-4597-a264-e60a82acd23f None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:44.869753 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 399/809] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 14:55:44 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 14:55:44.871890 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20977f2-8a0d-4209-9d07-d2df8ded1bac None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:44.900096 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab4d014-395e-4b7d-a333-e48ed6714292 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:44.902227 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab4d014-395e-4b7d-a333-e48ed6714292 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:44.903473 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab4d014-395e-4b7d-a333-e48ed6714292 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:44.903473 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab4d014-395e-4b7d-a333-e48ed6714292 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:44.924982 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab4d014-395e-4b7d-a333-e48ed6714292 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:44.936109 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab4d014-395e-4b7d-a333-e48ed6714292 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:44.970860 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-7ab4d014-395e-4b7d-a333-e48ed6714292 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 14:55:44.970860 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:55:44.970860 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:55:44.970860 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:55:44.970860 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:55:44.970860 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:55:44.970860 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:55:44.970860 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:55:44.970860 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:55:44.970860 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:55:44.970860 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:55:44.970860 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:55:44.970860 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 14:55:44.970860 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 14:55:44.970860 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 14:55:44.971841 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 14:55:44.971841 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:55:44.971841 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:55:44.971841 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:55:44.971841 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:55:44.971841 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:55:44.971841 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:55:44.971841 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:55:44.971841 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 26 14:55:44.971841 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:55:44.971841 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 26 14:55:44.971841 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:55:44.971841 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:55:44.971841 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:55:44.971841 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 26 14:55:44.971841 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:55:44.971841 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 26 14:55:44.973377 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 14:55:44.973377 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 14:55:44.973377 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 14:55:44.973377 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 14:55:44.973377 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 14:55:44.973377 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 14:55:44.973377 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 14:55:44.973377 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 26 14:55:44.973377 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 411/810] 10.222.0.46 () {56 vars in 1111 bytes} [Thu May 26 14:55:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:55:44.973930 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab4d014-395e-4b7d-a333-e48ed6714292 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:45.055379 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20977f2-8a0d-4209-9d07-d2df8ded1bac None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:45.100934 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:45.125117 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a37497-d126-4e0d-817f-13ff12ed2ba8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:45.126320 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a37497-d126-4e0d-817f-13ff12ed2ba8 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:45.126950 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a37497-d126-4e0d-817f-13ff12ed2ba8 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:45.127645 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a37497-d126-4e0d-817f-13ff12ed2ba8 None admin] PATH_INFO: `/v3/services` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:45.128636 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a37497-d126-4e0d-817f-13ff12ed2ba8 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:55:45.129264 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a37497-d126-4e0d-817f-13ff12ed2ba8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:45.131295 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a37497-d126-4e0d-817f-13ff12ed2ba8 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:45.137667 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab4d014-395e-4b7d-a333-e48ed6714292 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:45.151379 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 400/811] 10.222.0.46 () {56 vars in 1124 bytes} [Thu May 26 14:55:44 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:55:45.156254 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a37497-d126-4e0d-817f-13ff12ed2ba8 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:45.163436 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:45.217827 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cff41c1-8c15-4302-b852-06e6e4492fce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:45.218788 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cff41c1-8c15-4302-b852-06e6e4492fce None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:45.219264 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cff41c1-8c15-4302-b852-06e6e4492fce None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:45.219699 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cff41c1-8c15-4302-b852-06e6e4492fce None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:45.259122 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cff41c1-8c15-4302-b852-06e6e4492fce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 14:55:45.290408 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-6cff41c1-8c15-4302-b852-06e6e4492fce None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 14:55:45.290408 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:55:45.290408 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:55:45.290408 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:55:45.290408 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:55:45.290408 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:55:45.290408 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:55:45.290408 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:55:45.290408 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:55:45.290408 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:55:45.290408 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:55:45.290408 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:55:45.290408 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 14:55:45.290408 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 14:55:45.290408 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 14:55:45.290408 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 14:55:45.291462 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 14:55:45.291462 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 14:55:45.291462 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 14:55:45.291462 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 14:55:45.291462 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:55:45.291462 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:55:45.291462 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:55:45.291462 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:55:45.291462 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:55:45.291462 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:55:45.291462 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:55:45.291462 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 26 14:55:45.291462 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:55:45.291462 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 26 14:55:45.291462 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:55:45.291462 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:55:45.292331 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:55:45.292331 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 26 14:55:45.292331 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:55:45.292331 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 26 14:55:45.292331 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 14:55:45.292331 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 14:55:45.292331 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 14:55:45.292331 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 14:55:45.292331 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 14:55:45.292331 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 14:55:45.292331 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 14:55:45.292331 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 26 14:55:45.300169 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 412/812] 10.222.0.46 () {56 vars in 1096 bytes} [Thu May 26 14:55:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 320 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:55:45.301715 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cff41c1-8c15-4302-b852-06e6e4492fce None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:45.313339 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a37497-d126-4e0d-817f-13ff12ed2ba8 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:45.354431 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92b6f99-ac58-4ff6-a351-c79919b867ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:45.355455 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92b6f99-ac58-4ff6-a351-c79919b867ca None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:45.355940 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92b6f99-ac58-4ff6-a351-c79919b867ca None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:45.356520 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92b6f99-ac58-4ff6-a351-c79919b867ca None admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:45.357902 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92b6f99-ac58-4ff6-a351-c79919b867ca None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:55:45.358484 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92b6f99-ac58-4ff6-a351-c79919b867ca None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:45.360338 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92b6f99-ac58-4ff6-a351-c79919b867ca None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:45.378268 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 401/813] 10.222.0.46 () {56 vars in 1148 bytes} [Thu May 26 14:55:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:55:45.379850 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92b6f99-ac58-4ff6-a351-c79919b867ca None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:45.514595 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:45.539652 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cff41c1-8c15-4302-b852-06e6e4492fce None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:45.610655 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe854db6-6eba-46fd-9d4c-9f8f60c45c18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:45.612964 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe854db6-6eba-46fd-9d4c-9f8f60c45c18 None admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:45.615785 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe854db6-6eba-46fd-9d4c-9f8f60c45c18 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:45.617979 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe854db6-6eba-46fd-9d4c-9f8f60c45c18 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:45.622210 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe854db6-6eba-46fd-9d4c-9f8f60c45c18 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:45.625644 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe854db6-6eba-46fd-9d4c-9f8f60c45c18 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:45.663899 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92b6f99-ac58-4ff6-a351-c79919b867ca None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:45.688994 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 413/814] 10.222.0.46 () {60 vars in 1144 bytes} [Thu May 26 14:55:45 2022] POST /identity/v3/endpoints => generated 346 bytes in 388 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 14:55:45.694850 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe854db6-6eba-46fd-9d4c-9f8f60c45c18 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:45.779898 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:45.784643 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da197e3-a513-4e56-8d27-6cde502299a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:45.785405 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da197e3-a513-4e56-8d27-6cde502299a6 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:45.785824 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da197e3-a513-4e56-8d27-6cde502299a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:45.786083 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da197e3-a513-4e56-8d27-6cde502299a6 None admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:45.786866 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da197e3-a513-4e56-8d27-6cde502299a6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:55:45.789504 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da197e3-a513-4e56-8d27-6cde502299a6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:45.789504 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da197e3-a513-4e56-8d27-6cde502299a6 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:45.815123 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 402/815] 10.222.0.46 () {56 vars in 1099 bytes} [Thu May 26 14:55:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 436 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:55:45.824849 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da197e3-a513-4e56-8d27-6cde502299a6 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:46.001226 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe854db6-6eba-46fd-9d4c-9f8f60c45c18 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:46.002674 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da197e3-a513-4e56-8d27-6cde502299a6 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:46.025579 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:46.062498 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874d53f4-8d0e-4c89-8dcd-c3e7d89b30a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:46.064118 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874d53f4-8d0e-4c89-8dcd-c3e7d89b30a8 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:46.064759 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874d53f4-8d0e-4c89-8dcd-c3e7d89b30a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:46.065544 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874d53f4-8d0e-4c89-8dcd-c3e7d89b30a8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:46.081479 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9dfbbbb-9e69-4480-9b59-955172c3d8ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:46.082578 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dfbbbb-9e69-4480-9b59-955172c3d8ef None admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:46.083055 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dfbbbb-9e69-4480-9b59-955172c3d8ef None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:46.083569 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9dfbbbb-9e69-4480-9b59-955172c3d8ef None admin] PATH_INFO: `/v3/projects/a17fc8227d4c4da6b2847b350e912ee3/users/a4bea272624b45d8ac2ad27bd07df885/roles/a2e551773cbd4df084020676865b8222` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:46.084639 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874d53f4-8d0e-4c89-8dcd-c3e7d89b30a8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:46.086709 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9dfbbbb-9e69-4480-9b59-955172c3d8ef None admin] RBAC: Authorizing `identity:create_grant(project_id=a17fc8227d4c4da6b2847b350e912ee3, user_id=a4bea272624b45d8ac2ad27bd07df885, role_id=a2e551773cbd4df084020676865b8222)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:46.087842 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874d53f4-8d0e-4c89-8dcd-c3e7d89b30a8 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:46.089006 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9dfbbbb-9e69-4480-9b59-955172c3d8ef None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:46.118532 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-874d53f4-8d0e-4c89-8dcd-c3e7d89b30a8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 14:55:46.118532 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:55:46.118532 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:55:46.118532 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:55:46.118532 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:55:46.118532 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:55:46.118532 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:55:46.118532 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:55:46.118532 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:55:46.118532 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:55:46.118532 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:55:46.118532 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:55:46.118532 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 14:55:46.118532 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 14:55:46.118532 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 14:55:46.120092 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 14:55:46.120092 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:55:46.120092 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:55:46.120092 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:55:46.120092 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:55:46.120092 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:55:46.120092 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:55:46.120092 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:55:46.120092 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 26 14:55:46.120092 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:55:46.120092 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 26 14:55:46.120092 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:55:46.120092 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:55:46.120092 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:55:46.120092 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 26 14:55:46.120092 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:55:46.120092 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 26 14:55:46.120987 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 14:55:46.120987 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 14:55:46.120987 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 14:55:46.120987 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 26 14:55:46.123826 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 403/816] 10.222.0.46 () {56 vars in 1108 bytes} [Thu May 26 14:55:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 298 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:55:46.128039 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874d53f4-8d0e-4c89-8dcd-c3e7d89b30a8 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:46.134041 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 414/817] 10.222.0.46 () {58 vars in 1439 bytes} [Thu May 26 14:55:45 2022] PUT /identity/v3/projects/a17fc8227d4c4da6b2847b350e912ee3/users/a4bea272624b45d8ac2ad27bd07df885/roles/a2e551773cbd4df084020676865b8222 => generated 0 bytes in 444 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 14:55:46.258333 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874d53f4-8d0e-4c89-8dcd-c3e7d89b30a8 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:46.296917 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0a92a5-5262-4cad-a510-6ce598a64741 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:46.297724 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0a92a5-5262-4cad-a510-6ce598a64741 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:46.298071 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0a92a5-5262-4cad-a510-6ce598a64741 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:46.298410 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0a92a5-5262-4cad-a510-6ce598a64741 None admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:46.299219 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0a92a5-5262-4cad-a510-6ce598a64741 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:55:46.299586 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0a92a5-5262-4cad-a510-6ce598a64741 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:46.301090 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0a92a5-5262-4cad-a510-6ce598a64741 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:46.316078 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 404/818] 10.222.0.46 () {56 vars in 1109 bytes} [Thu May 26 14:55:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:55:46.327103 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9dfbbbb-9e69-4480-9b59-955172c3d8ef None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:46.513255 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:46.528375 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9dfbbbb-9e69-4480-9b59-955172c3d8ef None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:46.560147 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b186df-7627-4a03-bdac-978bc2c414fa None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:46.560874 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b186df-7627-4a03-bdac-978bc2c414fa None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:46.561379 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b186df-7627-4a03-bdac-978bc2c414fa None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:46.564358 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 405/819] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 14:55:46 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 14:55:46.572939 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71902ec-e988-4ff5-b679-14da7b3b144b None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:46.573763 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71902ec-e988-4ff5-b679-14da7b3b144b None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:46.574264 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71902ec-e988-4ff5-b679-14da7b3b144b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:46.597163 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-b71902ec-e988-4ff5-b679-14da7b3b144b None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:55:46.598720 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-b71902ec-e988-4ff5-b679-14da7b3b144b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:55:46.599963 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f163b6c-496f-4dbd-bc2b-dc5f9b78e490 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:46.601105 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f163b6c-496f-4dbd-bc2b-dc5f9b78e490 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:46.601791 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f163b6c-496f-4dbd-bc2b-dc5f9b78e490 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:46.602442 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f163b6c-496f-4dbd-bc2b-dc5f9b78e490 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:46.621942 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f163b6c-496f-4dbd-bc2b-dc5f9b78e490 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:46.624137 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f163b6c-496f-4dbd-bc2b-dc5f9b78e490 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:46.639189 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:46.657819 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-6f163b6c-496f-4dbd-bc2b-dc5f9b78e490 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 26 14:55:46.657819 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:55:46.657819 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:55:46.657819 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:55:46.657819 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:55:46.657819 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:55:46.657819 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:55:46.657819 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:55:46.657819 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:55:46.657819 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:55:46.657819 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:55:46.657819 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:55:46.657819 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 14:55:46.657819 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 14:55:46.657819 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 14:55:46.657819 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 14:55:46.659160 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:55:46.659160 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:55:46.659160 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 14:55:46.659160 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 14:55:46.659160 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 14:55:46.659160 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 14:55:46.659160 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:55:46.659160 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:55:46.659160 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:55:46.659160 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:55:46.659160 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:55:46.659160 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 26 14:55:46.659160 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:55:46.659160 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 26 14:55:46.659160 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:55:46.659160 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:55:46.659160 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:55:46.659160 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 26 14:55:46.660333 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:55:46.660333 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 26 14:55:46.660333 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 14:55:46.660333 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 14:55:46.660333 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 14:55:46.660333 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 14:55:46.660333 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 14:55:46.660333 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 14:55:46.660333 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 26 14:55:46.660333 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 26 14:55:46.665132 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 415/820] 10.222.0.46 () {56 vars in 1093 bytes} [Thu May 26 14:55:46 2022] GET /identity/v3/users/nova => generated 82 bytes in 338 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 14:55:46.670075 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f163b6c-496f-4dbd-bc2b-dc5f9b78e490 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:46.758136 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 406/821] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 14:55:46 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 14:55:46.806919 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71902ec-e988-4ff5-b679-14da7b3b144b None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:46.870409 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f163b6c-496f-4dbd-bc2b-dc5f9b78e490 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:46.909560 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b4aac8-c85e-4635-a5f3-8496eea851ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:46.910620 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b4aac8-c85e-4635-a5f3-8496eea851ac None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:46.911110 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b4aac8-c85e-4635-a5f3-8496eea851ac None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:46.911621 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b4aac8-c85e-4635-a5f3-8496eea851ac None admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:46.912678 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b4aac8-c85e-4635-a5f3-8496eea851ac None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:55:46.913064 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b4aac8-c85e-4635-a5f3-8496eea851ac None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:46.914868 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b4aac8-c85e-4635-a5f3-8496eea851ac None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:46.937647 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 416/822] 10.222.0.46 () {56 vars in 1133 bytes} [Thu May 26 14:55:46 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:55:46.947136 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b4aac8-c85e-4635-a5f3-8496eea851ac None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:46.968789 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:47.062548 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71902ec-e988-4ff5-b679-14da7b3b144b None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:47.104633 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb4d98d-24d4-4ef7-92a5-cc54a64b3775 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:47.106008 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb4d98d-24d4-4ef7-92a5-cc54a64b3775 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:47.106304 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb4d98d-24d4-4ef7-92a5-cc54a64b3775 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:47.106707 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb4d98d-24d4-4ef7-92a5-cc54a64b3775 None admin] PATH_INFO: `/v3/services/network` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:47.107380 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb4d98d-24d4-4ef7-92a5-cc54a64b3775 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:47.109320 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb4d98d-24d4-4ef7-92a5-cc54a64b3775 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:47.123819 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-0fb4d98d-24d4-4ef7-92a5-cc54a64b3775 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 26 14:55:47.123819 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:55:47.123819 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 14:55:47.123819 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 14:55:47.123819 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 14:55:47.123819 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 14:55:47.123819 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 14:55:47.123819 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 14:55:47.123819 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 26 14:55:47.123819 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 26 14:55:47.123819 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 14:55:47.123819 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 26 14:55:47.123819 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:55:47.123819 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:55:47.123819 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:55:47.123819 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:55:47.123819 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:55:47.125239 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:55:47.125239 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:55:47.125239 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:55:47.125239 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:55:47.125239 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:55:47.125239 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:55:47.125239 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 14:55:47.125239 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 14:55:47.125239 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 14:55:47.125239 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 14:55:47.125239 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:55:47.125239 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:55:47.125239 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:55:47.125239 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:55:47.125239 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:55:47.125239 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:55:47.126468 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:55:47.126468 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 26 14:55:47.126468 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:55:47.126468 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 26 14:55:47.126468 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:55:47.126468 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:55:47.126468 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:55:47.126468 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 26 14:55:47.126468 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:55:47.126468 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 26 14:55:47.126468 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 14:55:47.126468 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 14:55:47.126468 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 26 14:55:47.126468 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 26 14:55:47.126468 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 407/823] 10.222.0.46 () {56 vars in 1111 bytes} [Thu May 26 14:55:46 2022] GET /identity/v3/services/network => generated 88 bytes in 320 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 14:55:47.129891 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b4aac8-c85e-4635-a5f3-8496eea851ac None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:47.131508 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb4d98d-24d4-4ef7-92a5-cc54a64b3775 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:47.184631 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453aeb4c-2b12-4fa1-a5a4-18d70414b9da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:47.185184 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453aeb4c-2b12-4fa1-a5a4-18d70414b9da None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:47.185530 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453aeb4c-2b12-4fa1-a5a4-18d70414b9da None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:47.185842 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453aeb4c-2b12-4fa1-a5a4-18d70414b9da None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:47.193007 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:47.268665 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453aeb4c-2b12-4fa1-a5a4-18d70414b9da None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:47.270705 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453aeb4c-2b12-4fa1-a5a4-18d70414b9da None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:47.300343 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:47.421379 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 417/824] 10.222.0.46 () {58 vars in 1282 bytes} [Thu May 26 14:55:46 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 475 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 26 14:55:47.430079 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453aeb4c-2b12-4fa1-a5a4-18d70414b9da None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:47.558619 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453aeb4c-2b12-4fa1-a5a4-18d70414b9da None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:47.573543 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb4d98d-24d4-4ef7-92a5-cc54a64b3775 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:47.574245 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:47.618478 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5269f641-097c-421d-9eef-95bc8a53e018 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:47.619844 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5269f641-097c-421d-9eef-95bc8a53e018 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:47.620136 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5269f641-097c-421d-9eef-95bc8a53e018 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:47.620417 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5269f641-097c-421d-9eef-95bc8a53e018 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:47.664067 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5269f641-097c-421d-9eef-95bc8a53e018 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:47.666825 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5269f641-097c-421d-9eef-95bc8a53e018 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:47.691709 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67488d3f-bcc1-49c0-8b49-3c92f6af069a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:47.692549 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67488d3f-bcc1-49c0-8b49-3c92f6af069a None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:47.692840 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67488d3f-bcc1-49c0-8b49-3c92f6af069a None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:47.693119 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67488d3f-bcc1-49c0-8b49-3c92f6af069a None admin] PATH_INFO: `/v3/services` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:47.694064 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67488d3f-bcc1-49c0-8b49-3c92f6af069a None admin] RBAC: Adding query filter params (name=network) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:55:47.694436 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67488d3f-bcc1-49c0-8b49-3c92f6af069a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:47.696274 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67488d3f-bcc1-49c0-8b49-3c92f6af069a None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:47.709918 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 408/825] 10.222.0.46 () {56 vars in 1112 bytes} [Thu May 26 14:55:47 2022] GET /identity/v3/services?name=network => generated 124 bytes in 579 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:55:47.715635 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67488d3f-bcc1-49c0-8b49-3c92f6af069a None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:47.723029 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-5269f641-097c-421d-9eef-95bc8a53e018 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 14:55:47.723029 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:55:47.723029 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:55:47.723029 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:55:47.723029 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:55:47.723029 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:55:47.723029 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:55:47.723029 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:55:47.723029 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:55:47.723029 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:55:47.723029 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:55:47.723029 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:55:47.723029 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 14:55:47.723029 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 14:55:47.723029 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 14:55:47.723981 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 14:55:47.723981 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:55:47.723981 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:55:47.723981 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:55:47.723981 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:55:47.723981 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:55:47.723981 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:55:47.723981 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:55:47.723981 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 26 14:55:47.723981 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:55:47.723981 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 26 14:55:47.723981 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:55:47.723981 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:55:47.723981 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:55:47.723981 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 26 14:55:47.723981 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:55:47.723981 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 26 14:55:47.724870 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 14:55:47.724870 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 14:55:47.724870 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 14:55:47.724870 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 26 14:55:47.735919 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 418/826] 10.222.0.46 () {56 vars in 1108 bytes} [Thu May 26 14:55:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:55:47.736979 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5269f641-097c-421d-9eef-95bc8a53e018 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:47.877263 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:47.880430 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5269f641-097c-421d-9eef-95bc8a53e018 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:47.910083 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67488d3f-bcc1-49c0-8b49-3c92f6af069a None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:47.928025 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c2ef15-5214-4160-833d-fbee14329c2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:47.929967 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c2ef15-5214-4160-833d-fbee14329c2a None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:47.930611 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c2ef15-5214-4160-833d-fbee14329c2a None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:47.931364 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c2ef15-5214-4160-833d-fbee14329c2a None admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:47.932857 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c2ef15-5214-4160-833d-fbee14329c2a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:55:47.933507 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c2ef15-5214-4160-833d-fbee14329c2a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:47.936621 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c2ef15-5214-4160-833d-fbee14329c2a None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:47.957430 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671a9a9a-a2fe-49f1-8ac0-86f66432fa28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:47.958202 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671a9a9a-a2fe-49f1-8ac0-86f66432fa28 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:47.958624 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671a9a9a-a2fe-49f1-8ac0-86f66432fa28 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:47.958967 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671a9a9a-a2fe-49f1-8ac0-86f66432fa28 None admin] PATH_INFO: `/v3/services` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:47.959851 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671a9a9a-a2fe-49f1-8ac0-86f66432fa28 None admin] RBAC: Adding query filter params (type=network) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:55:47.960257 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671a9a9a-a2fe-49f1-8ac0-86f66432fa28 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:47.962180 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671a9a9a-a2fe-49f1-8ac0-86f66432fa28 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:47.963537 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 419/827] 10.222.0.46 () {56 vars in 1109 bytes} [Thu May 26 14:55:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:55:47.965701 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6d0865-1754-4850-854d-b146d4cf372f None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:47.966416 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6d0865-1754-4850-854d-b146d4cf372f None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:47.966973 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6d0865-1754-4850-854d-b146d4cf372f None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:47.970260 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 420/828] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 14:55:47 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 14:55:47.972345 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6d0865-1754-4850-854d-b146d4cf372f None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:47.973978 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 409/829] 10.222.0.46 () {56 vars in 1112 bytes} [Thu May 26 14:55:47 2022] GET /identity/v3/services?type=network => generated 350 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 14:55:47.980183 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6baefc-1086-4efb-b84b-e32c61e0f2be None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:47.980612 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6baefc-1086-4efb-b84b-e32c61e0f2be None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:47.981000 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6baefc-1086-4efb-b84b-e32c61e0f2be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:48.001220 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-ce6baefc-1086-4efb-b84b-e32c61e0f2be None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:55:48.002596 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-ce6baefc-1086-4efb-b84b-e32c61e0f2be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:55:48.016256 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:48.083069 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 410/830] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 14:55:47 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 14:55:48.084285 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6baefc-1086-4efb-b84b-e32c61e0f2be None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:48.145266 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6d0865-1754-4850-854d-b146d4cf372f None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:48.178990 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:48.187553 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edc7a96-a743-462f-9834-6ba9bb85e9db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:48.188737 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edc7a96-a743-462f-9834-6ba9bb85e9db None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:48.189240 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edc7a96-a743-462f-9834-6ba9bb85e9db None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:48.189759 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3edc7a96-a743-462f-9834-6ba9bb85e9db None admin] PATH_INFO: `/v3/projects/service` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:48.197837 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3edc7a96-a743-462f-9834-6ba9bb85e9db None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:48.199477 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3edc7a96-a743-462f-9834-6ba9bb85e9db None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:48.214629 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6baefc-1086-4efb-b84b-e32c61e0f2be None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:48.221768 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-3edc7a96-a743-462f-9834-6ba9bb85e9db None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 14:55:48.221768 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:55:48.221768 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:55:48.221768 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:55:48.221768 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:55:48.221768 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:55:48.221768 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:55:48.221768 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:55:48.221768 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:55:48.221768 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:55:48.221768 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:55:48.221768 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:55:48.221768 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 14:55:48.221768 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 14:55:48.221768 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 14:55:48.222801 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 14:55:48.222801 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:55:48.222801 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:55:48.222801 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:55:48.222801 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:55:48.222801 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:55:48.222801 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:55:48.222801 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:55:48.222801 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 26 14:55:48.222801 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:55:48.222801 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 26 14:55:48.222801 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:55:48.222801 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:55:48.222801 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:55:48.222801 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 26 14:55:48.222801 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:55:48.222801 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 26 14:55:48.224135 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 14:55:48.224135 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 14:55:48.224135 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 14:55:48.224135 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 14:55:48.224135 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 14:55:48.224135 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 14:55:48.224135 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 14:55:48.224135 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 26 14:55:48.224135 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 421/831] 10.222.0.46 () {56 vars in 1111 bytes} [Thu May 26 14:55:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:55:48.229544 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edc7a96-a743-462f-9834-6ba9bb85e9db None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:48.253926 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021bd6d6-2684-4071-8515-3ab451a737ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:48.255048 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021bd6d6-2684-4071-8515-3ab451a737ee None admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:48.255336 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021bd6d6-2684-4071-8515-3ab451a737ee None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:48.255612 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021bd6d6-2684-4071-8515-3ab451a737ee None admin] PATH_INFO: `/v3/endpoints` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:48.256360 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021bd6d6-2684-4071-8515-3ab451a737ee None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:48.257938 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-021bd6d6-2684-4071-8515-3ab451a737ee None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:48.282148 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 411/832] 10.222.0.46 () {60 vars in 1144 bytes} [Thu May 26 14:55:48 2022] POST /identity/v3/endpoints => generated 327 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 14:55:48.283489 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021bd6d6-2684-4071-8515-3ab451a737ee None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:48.328348 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:48.370092 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3edc7a96-a743-462f-9834-6ba9bb85e9db None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:48.477176 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-021bd6d6-2684-4071-8515-3ab451a737ee None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:48.486205 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c1fe95-90c4-43d8-bbe7-bea78d250fa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:48.494338 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c1fe95-90c4-43d8-bbe7-bea78d250fa2 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:48.495087 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c1fe95-90c4-43d8-bbe7-bea78d250fa2 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:48.495717 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c1fe95-90c4-43d8-bbe7-bea78d250fa2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:48.526456 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c1fe95-90c4-43d8-bbe7-bea78d250fa2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 14:55:48.538551 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-86c1fe95-90c4-43d8-bbe7-bea78d250fa2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 26 14:55:48.538551 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:55:48.538551 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:55:48.538551 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:55:48.538551 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:55:48.538551 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:55:48.538551 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:55:48.538551 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:55:48.538551 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:55:48.538551 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:55:48.538551 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:55:48.538551 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:55:48.538551 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 14:55:48.538551 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 14:55:48.538551 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 14:55:48.538551 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 14:55:48.539532 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 14:55:48.539532 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 14:55:48.539532 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 14:55:48.539532 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 14:55:48.539532 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:55:48.539532 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:55:48.539532 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:55:48.539532 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:55:48.539532 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:55:48.539532 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:55:48.539532 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:55:48.539532 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 26 14:55:48.539532 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:55:48.539532 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 26 14:55:48.539532 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:55:48.539532 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:55:48.540387 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:55:48.540387 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 26 14:55:48.540387 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:55:48.540387 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 26 14:55:48.540387 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 14:55:48.540387 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 14:55:48.540387 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 14:55:48.540387 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 14:55:48.540387 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 14:55:48.540387 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 14:55:48.540387 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 26 14:55:48.540387 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 26 14:55:48.543806 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 422/833] 10.222.0.46 () {56 vars in 1102 bytes} [Thu May 26 14:55:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 319 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:55:48.545330 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b210482a-b7ba-4d7b-9e04-85c832802f0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:48.546126 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b210482a-b7ba-4d7b-9e04-85c832802f0f None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:48.546486 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b210482a-b7ba-4d7b-9e04-85c832802f0f None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:48.546834 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b210482a-b7ba-4d7b-9e04-85c832802f0f None admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:48.547699 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b210482a-b7ba-4d7b-9e04-85c832802f0f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:55:48.548277 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b210482a-b7ba-4d7b-9e04-85c832802f0f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:48.549148 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c1fe95-90c4-43d8-bbe7-bea78d250fa2 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:48.551092 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b210482a-b7ba-4d7b-9e04-85c832802f0f None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:48.567434 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 412/834] 10.222.0.46 () {56 vars in 1148 bytes} [Thu May 26 14:55:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:55:48.569758 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:48.573499 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b210482a-b7ba-4d7b-9e04-85c832802f0f None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:48.715830 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c1fe95-90c4-43d8-bbe7-bea78d250fa2 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:48.719418 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b210482a-b7ba-4d7b-9e04-85c832802f0f None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:48.756051 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ccbdc95-8531-462b-bb51-ec197927a797 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:48.756632 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccbdc95-8531-462b-bb51-ec197927a797 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:48.756914 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccbdc95-8531-462b-bb51-ec197927a797 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:48.757188 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ccbdc95-8531-462b-bb51-ec197927a797 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:48.758046 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccbdc95-8531-462b-bb51-ec197927a797 None admin] RBAC: Adding query filter params (role.id=a2e551773cbd4df084020676865b8222, scope.project.id=a17fc8227d4c4da6b2847b350e912ee3, user.id=d6639353c70449d1b3d10cd6487c3b00) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:55:48.758363 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccbdc95-8531-462b-bb51-ec197927a797 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:48.760347 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ccbdc95-8531-462b-bb51-ec197927a797 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:48.761161 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698d2750-a498-4513-ac32-e45a2882fe4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:48.762269 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698d2750-a498-4513-ac32-e45a2882fe4f None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:48.762750 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698d2750-a498-4513-ac32-e45a2882fe4f None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:48.763152 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698d2750-a498-4513-ac32-e45a2882fe4f None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:48.764181 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698d2750-a498-4513-ac32-e45a2882fe4f None admin] RBAC: Adding query filter params (name=service) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:55:48.764667 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698d2750-a498-4513-ac32-e45a2882fe4f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:48.766842 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698d2750-a498-4513-ac32-e45a2882fe4f None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:48.769794 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 413/835] 10.222.0.46 () {56 vars in 1374 bytes} [Thu May 26 14:55:48 2022] GET /identity/v3/role_assignments?user.id=d6639353c70449d1b3d10cd6487c3b00&scope.project.id=a17fc8227d4c4da6b2847b350e912ee3&role.id=a2e551773cbd4df084020676865b8222 => generated 608 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:55:48.772043 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e96a78b-c7c2-4be3-9d1b-1527d4f4ee98 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:48.772667 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e96a78b-c7c2-4be3-9d1b-1527d4f4ee98 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:48.773190 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e96a78b-c7c2-4be3-9d1b-1527d4f4ee98 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:48.780842 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 423/836] 10.222.0.46 () {56 vars in 1103 bytes} [Thu May 26 14:55:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 14:55:48.781433 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 414/837] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 14:55:48 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 14:55:48.783729 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c058ba37-9b9f-4bdd-ac27-6018c829152a None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:48.784247 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e96a78b-c7c2-4be3-9d1b-1527d4f4ee98 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:48.784358 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c058ba37-9b9f-4bdd-ac27-6018c829152a None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:48.784649 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c058ba37-9b9f-4bdd-ac27-6018c829152a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:48.792468 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:48.813719 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-c058ba37-9b9f-4bdd-ac27-6018c829152a None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:55:48.815629 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-c058ba37-9b9f-4bdd-ac27-6018c829152a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:55:48.879959 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:48.949477 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 424/838] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 14:55:48 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 14:55:48.964755 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e96a78b-c7c2-4be3-9d1b-1527d4f4ee98 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:48.967918 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c058ba37-9b9f-4bdd-ac27-6018c829152a None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:49.012489 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450bc076-b68d-45ec-9654-7810dbb55b4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:49.013320 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450bc076-b68d-45ec-9654-7810dbb55b4a None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:49.013824 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450bc076-b68d-45ec-9654-7810dbb55b4a None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:49.014175 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450bc076-b68d-45ec-9654-7810dbb55b4a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:49.029052 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450bc076-b68d-45ec-9654-7810dbb55b4a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:49.030569 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450bc076-b68d-45ec-9654-7810dbb55b4a None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:49.053485 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-450bc076-b68d-45ec-9654-7810dbb55b4a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 14:55:49.053485 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:55:49.053485 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:55:49.053485 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:55:49.053485 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:55:49.053485 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:55:49.053485 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:55:49.053485 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:55:49.053485 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:55:49.053485 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:55:49.053485 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:55:49.053485 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:55:49.053485 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 14:55:49.053485 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 14:55:49.053485 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 14:55:49.054440 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 14:55:49.054440 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:55:49.054440 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:55:49.054440 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:55:49.054440 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:55:49.054440 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:55:49.054440 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:55:49.054440 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:55:49.054440 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 26 14:55:49.054440 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:55:49.054440 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 26 14:55:49.054440 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:55:49.054440 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:55:49.054440 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:55:49.054440 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 26 14:55:49.054440 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:55:49.054440 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 26 14:55:49.055323 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 14:55:49.055323 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 14:55:49.055323 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 14:55:49.055323 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 26 14:55:49.058217 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 415/839] 10.222.0.46 () {56 vars in 1108 bytes} [Thu May 26 14:55:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 14:55:49.063345 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450bc076-b68d-45ec-9654-7810dbb55b4a None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:49.095231 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c058ba37-9b9f-4bdd-ac27-6018c829152a None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:49.129049 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:49.135491 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632d73e0-edb5-4457-9a33-c2d2ec0a2b44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:49.136246 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632d73e0-edb5-4457-9a33-c2d2ec0a2b44 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:49.136864 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632d73e0-edb5-4457-9a33-c2d2ec0a2b44 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:49.137682 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632d73e0-edb5-4457-9a33-c2d2ec0a2b44 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:49.138061 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632d73e0-edb5-4457-9a33-c2d2ec0a2b44 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:49.139660 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632d73e0-edb5-4457-9a33-c2d2ec0a2b44 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:49.153557 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-632d73e0-edb5-4457-9a33-c2d2ec0a2b44 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 26 14:55:49.153557 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:55:49.153557 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 14:55:49.153557 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 14:55:49.153557 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 14:55:49.153557 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 14:55:49.153557 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 14:55:49.153557 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 14:55:49.153557 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 26 14:55:49.153557 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 26 14:55:49.153557 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 14:55:49.153557 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 26 14:55:49.153557 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:55:49.153557 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:55:49.153557 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:55:49.153557 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:55:49.153557 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:55:49.154790 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:55:49.154790 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:55:49.154790 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:55:49.154790 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:55:49.154790 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:55:49.154790 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:55:49.154790 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 14:55:49.154790 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 14:55:49.154790 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 14:55:49.154790 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 14:55:49.154790 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:55:49.154790 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:55:49.154790 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:55:49.154790 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:55:49.154790 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:55:49.154790 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:55:49.155642 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:55:49.155642 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 26 14:55:49.155642 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:55:49.155642 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 26 14:55:49.155642 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:55:49.155642 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:55:49.155642 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:55:49.155642 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 26 14:55:49.155642 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:55:49.155642 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 26 14:55:49.155642 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 14:55:49.155642 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 14:55:49.155642 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 26 14:55:49.155642 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 26 14:55:49.161004 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 425/840] 10.222.0.46 () {56 vars in 1114 bytes} [Thu May 26 14:55:48 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 14:55:49.162097 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632d73e0-edb5-4457-9a33-c2d2ec0a2b44 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:49.191947 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450bc076-b68d-45ec-9654-7810dbb55b4a None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:49.206032 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:49.232866 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfaf8f65-b65f-49df-8de2-debae1f7d9de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:49.233963 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfaf8f65-b65f-49df-8de2-debae1f7d9de None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:49.234531 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfaf8f65-b65f-49df-8de2-debae1f7d9de None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:49.235013 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfaf8f65-b65f-49df-8de2-debae1f7d9de None admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:49.236042 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfaf8f65-b65f-49df-8de2-debae1f7d9de None admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:55:49.236561 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfaf8f65-b65f-49df-8de2-debae1f7d9de None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:49.238263 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfaf8f65-b65f-49df-8de2-debae1f7d9de None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:49.251348 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 416/841] 10.222.0.46 () {56 vars in 1109 bytes} [Thu May 26 14:55:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 14:55:49.259358 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfaf8f65-b65f-49df-8de2-debae1f7d9de None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:49.281229 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632d73e0-edb5-4457-9a33-c2d2ec0a2b44 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:49.319469 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdaf6527-f257-49b8-9740-12087b53e30c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:49.320148 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdaf6527-f257-49b8-9740-12087b53e30c None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:49.320426 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdaf6527-f257-49b8-9740-12087b53e30c None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:49.320697 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdaf6527-f257-49b8-9740-12087b53e30c None admin] PATH_INFO: `/v3/services` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:49.321494 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdaf6527-f257-49b8-9740-12087b53e30c None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:55:49.322352 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdaf6527-f257-49b8-9740-12087b53e30c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:49.324134 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdaf6527-f257-49b8-9740-12087b53e30c None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:49.330152 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 426/842] 10.222.0.46 () {56 vars in 1114 bytes} [Thu May 26 14:55:49 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:55:49.330962 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea47815-bcf8-495c-9857-3416a9e5196f None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:49.331109 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea47815-bcf8-495c-9857-3416a9e5196f None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:49.333843 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea47815-bcf8-495c-9857-3416a9e5196f None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:49.333843 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 427/843] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 14:55:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 14:55:49.339036 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea47815-bcf8-495c-9857-3416a9e5196f None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:49.392102 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfaf8f65-b65f-49df-8de2-debae1f7d9de None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:49.413538 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:49.439317 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed2ac4e-4c90-4848-a4da-050b04d94302 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:49.440264 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed2ac4e-4c90-4848-a4da-050b04d94302 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:49.440555 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed2ac4e-4c90-4848-a4da-050b04d94302 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:49.440957 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed2ac4e-4c90-4848-a4da-050b04d94302 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:49.450424 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed2ac4e-4c90-4848-a4da-050b04d94302 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:49.451786 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ed2ac4e-4c90-4848-a4da-050b04d94302 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:49.464012 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea47815-bcf8-495c-9857-3416a9e5196f None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:49.467231 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-7ed2ac4e-4c90-4848-a4da-050b04d94302 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 26 14:55:49.467231 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:55:49.467231 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:55:49.467231 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:55:49.467231 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:55:49.467231 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:55:49.467231 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:55:49.467231 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:55:49.467231 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:55:49.467231 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:55:49.467231 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:55:49.467231 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:55:49.467231 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 14:55:49.467231 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 14:55:49.467231 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 14:55:49.467231 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 14:55:49.468202 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:55:49.468202 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:55:49.468202 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 14:55:49.468202 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 14:55:49.468202 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 14:55:49.468202 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 14:55:49.468202 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:55:49.468202 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:55:49.468202 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:55:49.468202 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:55:49.468202 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:55:49.468202 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 26 14:55:49.468202 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:55:49.468202 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 26 14:55:49.468202 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:55:49.468202 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:55:49.468202 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:55:49.468202 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 26 14:55:49.469168 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:55:49.469168 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 26 14:55:49.469168 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 14:55:49.469168 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 14:55:49.469168 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 14:55:49.469168 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 14:55:49.469168 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 14:55:49.469168 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 14:55:49.469168 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 26 14:55:49.469168 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 26 14:55:49.469168 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 417/844] 10.222.0.46 () {56 vars in 1117 bytes} [Thu May 26 14:55:49 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 14:55:49.470266 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d819dbf-ce96-448a-ac89-53f38f50a4e0 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:49.470431 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d819dbf-ce96-448a-ac89-53f38f50a4e0 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:49.470699 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d819dbf-ce96-448a-ac89-53f38f50a4e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:49.488004 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-5d819dbf-ce96-448a-ac89-53f38f50a4e0 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:55:49.489197 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-5d819dbf-ce96-448a-ac89-53f38f50a4e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:55:49.508097 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a453f9-7dfa-47df-897c-66fcd87b7573 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:49.508767 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a453f9-7dfa-47df-897c-66fcd87b7573 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:49.509056 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a453f9-7dfa-47df-897c-66fcd87b7573 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:49.510884 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:49.511116 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a453f9-7dfa-47df-897c-66fcd87b7573 None admin] PATH_INFO: `/v3/services` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:49.511824 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a453f9-7dfa-47df-897c-66fcd87b7573 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:55:49.512138 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a453f9-7dfa-47df-897c-66fcd87b7573 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:49.513941 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a453f9-7dfa-47df-897c-66fcd87b7573 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:49.519015 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 428/845] 10.222.0.46 () {56 vars in 1114 bytes} [Thu May 26 14:55:49 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 14:55:49.519736 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a453f9-7dfa-47df-897c-66fcd87b7573 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:49.613131 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 418/846] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 14:55:49 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 14:55:49.637927 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d819dbf-ce96-448a-ac89-53f38f50a4e0 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:49.667090 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a453f9-7dfa-47df-897c-66fcd87b7573 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:49.705864 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1521408-d311-47b1-9806-9ec3add29716 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:49.706605 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1521408-d311-47b1-9806-9ec3add29716 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:49.706898 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1521408-d311-47b1-9806-9ec3add29716 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:49.707182 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1521408-d311-47b1-9806-9ec3add29716 None admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:49.708027 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1521408-d311-47b1-9806-9ec3add29716 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:55:49.708339 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1521408-d311-47b1-9806-9ec3add29716 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:49.709845 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1521408-d311-47b1-9806-9ec3add29716 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:49.714008 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:49.735126 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 429/847] 10.222.0.46 () {56 vars in 1149 bytes} [Thu May 26 14:55:49 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:55:49.740667 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1521408-d311-47b1-9806-9ec3add29716 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:49.783295 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d819dbf-ce96-448a-ac89-53f38f50a4e0 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:49.825708 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afaedbd6-53d5-4445-8e47-268352af67c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:49.826630 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afaedbd6-53d5-4445-8e47-268352af67c0 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:49.827003 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afaedbd6-53d5-4445-8e47-268352af67c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:49.827360 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afaedbd6-53d5-4445-8e47-268352af67c0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:49.838758 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afaedbd6-53d5-4445-8e47-268352af67c0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 14:55:49.846895 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-afaedbd6-53d5-4445-8e47-268352af67c0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 14:55:49.846895 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:55:49.846895 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:55:49.846895 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:55:49.846895 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:55:49.846895 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:55:49.846895 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:55:49.846895 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:55:49.846895 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:55:49.846895 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:55:49.846895 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:55:49.846895 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:55:49.846895 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 14:55:49.846895 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 14:55:49.846895 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 14:55:49.846895 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 14:55:49.847875 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 14:55:49.847875 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 14:55:49.847875 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 14:55:49.847875 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 14:55:49.847875 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:55:49.847875 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:55:49.847875 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:55:49.847875 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:55:49.847875 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:55:49.847875 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:55:49.847875 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:55:49.847875 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 26 14:55:49.847875 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:55:49.847875 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 26 14:55:49.847875 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:55:49.847875 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:55:49.848890 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:55:49.848890 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 26 14:55:49.848890 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:55:49.848890 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 26 14:55:49.848890 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 14:55:49.848890 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 14:55:49.848890 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 14:55:49.848890 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 14:55:49.848890 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 14:55:49.848890 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 14:55:49.848890 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 14:55:49.848890 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 26 14:55:49.848890 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 419/848] 10.222.0.46 () {56 vars in 1096 bytes} [Thu May 26 14:55:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 14:55:49.858941 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afaedbd6-53d5-4445-8e47-268352af67c0 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:49.862701 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1521408-d311-47b1-9806-9ec3add29716 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:49.891107 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:49.911301 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ebb40a-cf49-474f-8b42-8328995d6efb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:49.912663 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ebb40a-cf49-474f-8b42-8328995d6efb None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:49.913361 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ebb40a-cf49-474f-8b42-8328995d6efb None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:49.913884 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ebb40a-cf49-474f-8b42-8328995d6efb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:49.998632 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:50.011776 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afaedbd6-53d5-4445-8e47-268352af67c0 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:50.020295 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ebb40a-cf49-474f-8b42-8328995d6efb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:50.021790 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ebb40a-cf49-474f-8b42-8328995d6efb None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:50.053309 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a6a4c6-b98b-4e7a-9144-d3c2557b3364 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:50.054370 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a6a4c6-b98b-4e7a-9144-d3c2557b3364 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:50.054826 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a6a4c6-b98b-4e7a-9144-d3c2557b3364 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:50.055128 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a6a4c6-b98b-4e7a-9144-d3c2557b3364 None admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:50.055932 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a6a4c6-b98b-4e7a-9144-d3c2557b3364 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:55:50.056783 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a6a4c6-b98b-4e7a-9144-d3c2557b3364 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:50.058200 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a6a4c6-b98b-4e7a-9144-d3c2557b3364 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:50.069935 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 420/849] 10.222.0.46 () {56 vars in 1099 bytes} [Thu May 26 14:55:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:55:50.075609 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a6a4c6-b98b-4e7a-9144-d3c2557b3364 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:50.161640 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 430/850] 10.222.0.46 () {58 vars in 1282 bytes} [Thu May 26 14:55:49 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 26 14:55:50.166495 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ebb40a-cf49-474f-8b42-8328995d6efb None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:50.195996 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:50.221249 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a6a4c6-b98b-4e7a-9144-d3c2557b3364 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:50.257488 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb039af5-c1c4-4df2-8c74-82739f6b9883 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:50.259277 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb039af5-c1c4-4df2-8c74-82739f6b9883 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:50.259638 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb039af5-c1c4-4df2-8c74-82739f6b9883 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:50.259987 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb039af5-c1c4-4df2-8c74-82739f6b9883 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:50.271620 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:50.274256 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb039af5-c1c4-4df2-8c74-82739f6b9883 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:50.276356 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb039af5-c1c4-4df2-8c74-82739f6b9883 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:50.293226 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ebb40a-cf49-474f-8b42-8328995d6efb None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:50.299914 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-cb039af5-c1c4-4df2-8c74-82739f6b9883 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 14:55:50.299914 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:55:50.299914 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:55:50.299914 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:55:50.299914 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:55:50.299914 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:55:50.299914 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:55:50.299914 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:55:50.299914 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:55:50.299914 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:55:50.299914 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:55:50.299914 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:55:50.299914 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 14:55:50.299914 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 14:55:50.299914 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 14:55:50.301602 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 14:55:50.301602 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:55:50.301602 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:55:50.301602 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:55:50.301602 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:55:50.301602 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:55:50.301602 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:55:50.301602 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:55:50.301602 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 26 14:55:50.301602 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:55:50.301602 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 26 14:55:50.301602 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:55:50.301602 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:55:50.301602 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:55:50.301602 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 26 14:55:50.301602 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:55:50.301602 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 26 14:55:50.302868 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 14:55:50.302868 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 14:55:50.302868 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 14:55:50.302868 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 26 14:55:50.302868 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 421/851] 10.222.0.46 () {56 vars in 1108 bytes} [Thu May 26 14:55:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 14:55:50.308511 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb039af5-c1c4-4df2-8c74-82739f6b9883 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:50.332584 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151fca0c-a57d-47c8-8302-1bc8c9aa1689 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:50.333567 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151fca0c-a57d-47c8-8302-1bc8c9aa1689 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:50.333851 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151fca0c-a57d-47c8-8302-1bc8c9aa1689 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:50.334126 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151fca0c-a57d-47c8-8302-1bc8c9aa1689 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:50.348399 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151fca0c-a57d-47c8-8302-1bc8c9aa1689 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:50.349958 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151fca0c-a57d-47c8-8302-1bc8c9aa1689 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:50.369522 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-151fca0c-a57d-47c8-8302-1bc8c9aa1689 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 14:55:50.369522 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:55:50.369522 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:55:50.369522 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:55:50.369522 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:55:50.369522 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:55:50.369522 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:55:50.369522 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:55:50.369522 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:55:50.369522 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:55:50.369522 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:55:50.369522 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:55:50.369522 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 14:55:50.369522 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 14:55:50.369522 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 14:55:50.370478 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 14:55:50.370478 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:55:50.370478 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:55:50.370478 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:55:50.370478 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:55:50.370478 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:55:50.370478 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:55:50.370478 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:55:50.370478 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 26 14:55:50.370478 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:55:50.370478 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 26 14:55:50.370478 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:55:50.370478 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:55:50.370478 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:55:50.370478 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 26 14:55:50.370478 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:55:50.370478 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 26 14:55:50.371368 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 14:55:50.371368 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 14:55:50.371368 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 14:55:50.371368 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 26 14:55:50.375375 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 431/852] 10.222.0.46 () {56 vars in 1108 bytes} [Thu May 26 14:55:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 14:55:50.376188 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151fca0c-a57d-47c8-8302-1bc8c9aa1689 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:50.448392 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb039af5-c1c4-4df2-8c74-82739f6b9883 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:50.486822 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:50.495652 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151fca0c-a57d-47c8-8302-1bc8c9aa1689 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:50.502849 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0800c8e-3923-42ae-aa12-dc6d11aab33e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:50.504605 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0800c8e-3923-42ae-aa12-dc6d11aab33e None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:50.505117 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0800c8e-3923-42ae-aa12-dc6d11aab33e None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:50.505711 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0800c8e-3923-42ae-aa12-dc6d11aab33e None admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:50.507097 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0800c8e-3923-42ae-aa12-dc6d11aab33e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:55:50.507614 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0800c8e-3923-42ae-aa12-dc6d11aab33e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:50.510374 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0800c8e-3923-42ae-aa12-dc6d11aab33e None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:50.528155 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 422/853] 10.222.0.46 () {56 vars in 1109 bytes} [Thu May 26 14:55:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:55:50.532616 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea844214-029d-4a1c-97a5-1a2185c4ca2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:50.533314 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea844214-029d-4a1c-97a5-1a2185c4ca2b None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:50.533735 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea844214-029d-4a1c-97a5-1a2185c4ca2b None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:50.533969 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0800c8e-3923-42ae-aa12-dc6d11aab33e None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:50.534185 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea844214-029d-4a1c-97a5-1a2185c4ca2b None admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:50.535003 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea844214-029d-4a1c-97a5-1a2185c4ca2b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:55:50.535405 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea844214-029d-4a1c-97a5-1a2185c4ca2b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:50.536843 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea844214-029d-4a1c-97a5-1a2185c4ca2b None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:50.539829 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:50.557165 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 432/854] 10.222.0.46 () {56 vars in 1109 bytes} [Thu May 26 14:55:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:55:50.558171 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea844214-029d-4a1c-97a5-1a2185c4ca2b None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:50.669552 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0800c8e-3923-42ae-aa12-dc6d11aab33e None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:50.684876 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea844214-029d-4a1c-97a5-1a2185c4ca2b None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:50.710124 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395ef1f9-15c5-4e29-87b6-aa755774432a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:50.711096 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395ef1f9-15c5-4e29-87b6-aa755774432a None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:50.711486 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395ef1f9-15c5-4e29-87b6-aa755774432a None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:50.711959 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395ef1f9-15c5-4e29-87b6-aa755774432a None admin] PATH_INFO: `/v3/users/swift` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:50.720035 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395ef1f9-15c5-4e29-87b6-aa755774432a None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:50.721695 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395ef1f9-15c5-4e29-87b6-aa755774432a None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:50.736874 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e96c7e9-b674-43d3-9245-6c4a9d8db156 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:50.738273 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e96c7e9-b674-43d3-9245-6c4a9d8db156 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:50.738696 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e96c7e9-b674-43d3-9245-6c4a9d8db156 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:50.739098 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-395ef1f9-15c5-4e29-87b6-aa755774432a None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 26 14:55:50.739098 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:55:50.739098 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:55:50.739098 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:55:50.739098 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:55:50.739098 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:55:50.739098 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:55:50.739098 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:55:50.739098 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:55:50.739098 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:55:50.739098 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:55:50.739098 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:55:50.739098 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 14:55:50.739098 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 14:55:50.739098 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 14:55:50.739098 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 14:55:50.740073 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:55:50.740073 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:55:50.740073 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 14:55:50.740073 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 14:55:50.740073 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 14:55:50.740073 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 14:55:50.740073 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:55:50.740073 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:55:50.740073 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:55:50.740073 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:55:50.740073 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:55:50.740073 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 26 14:55:50.740073 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:55:50.740073 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 26 14:55:50.740073 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:55:50.740073 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:55:50.740073 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:55:50.740073 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 26 14:55:50.741013 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:55:50.741013 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 26 14:55:50.741013 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 14:55:50.741013 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 14:55:50.741013 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 14:55:50.741013 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 14:55:50.741013 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 14:55:50.741013 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 14:55:50.741013 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 26 14:55:50.741013 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 26 14:55:50.741013 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 423/855] 10.222.0.46 () {56 vars in 1096 bytes} [Thu May 26 14:55:50 2022] GET /identity/v3/users/swift => generated 83 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:55:50.741626 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e96c7e9-b674-43d3-9245-6c4a9d8db156 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:50.749778 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e96c7e9-b674-43d3-9245-6c4a9d8db156 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:50.750441 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395ef1f9-15c5-4e29-87b6-aa755774432a None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:50.751723 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e96c7e9-b674-43d3-9245-6c4a9d8db156 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:50.773753 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-1e96c7e9-b674-43d3-9245-6c4a9d8db156 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 14:55:50.773753 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:55:50.773753 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:55:50.773753 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:55:50.773753 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:55:50.773753 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:55:50.773753 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:55:50.773753 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:55:50.773753 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:55:50.773753 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:55:50.773753 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:55:50.773753 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:55:50.773753 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 14:55:50.773753 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 14:55:50.773753 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 14:55:50.774715 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 14:55:50.774715 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:55:50.774715 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:55:50.774715 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:55:50.774715 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:55:50.774715 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:55:50.774715 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:55:50.774715 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:55:50.774715 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 26 14:55:50.774715 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:55:50.774715 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 26 14:55:50.774715 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:55:50.774715 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:55:50.774715 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:55:50.774715 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 26 14:55:50.774715 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:55:50.774715 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 26 14:55:50.775683 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 14:55:50.775683 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 14:55:50.775683 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 14:55:50.775683 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 14:55:50.775683 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 14:55:50.775683 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 14:55:50.775683 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 14:55:50.775683 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 26 14:55:50.778721 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 433/856] 10.222.0.46 () {56 vars in 1111 bytes} [Thu May 26 14:55:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:55:50.793477 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e96c7e9-b674-43d3-9245-6c4a9d8db156 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:50.810763 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:50.869898 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:50.876690 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395ef1f9-15c5-4e29-87b6-aa755774432a None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:50.901712 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e96c7e9-b674-43d3-9245-6c4a9d8db156 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:50.922536 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10501051-4905-48ca-afba-8da4c3b0e088 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:50.923509 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10501051-4905-48ca-afba-8da4c3b0e088 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:50.924143 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10501051-4905-48ca-afba-8da4c3b0e088 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:50.924533 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10501051-4905-48ca-afba-8da4c3b0e088 None admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:50.925865 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10501051-4905-48ca-afba-8da4c3b0e088 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:55:50.926279 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10501051-4905-48ca-afba-8da4c3b0e088 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:50.928393 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10501051-4905-48ca-afba-8da4c3b0e088 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:50.945773 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d94f1b-6bf4-4d84-b33a-9b8955f1ddad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:50.946447 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d94f1b-6bf4-4d84-b33a-9b8955f1ddad None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:50.946769 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d94f1b-6bf4-4d84-b33a-9b8955f1ddad None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:50.947052 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d94f1b-6bf4-4d84-b33a-9b8955f1ddad None admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:50.948463 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d94f1b-6bf4-4d84-b33a-9b8955f1ddad None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:55:50.948783 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d94f1b-6bf4-4d84-b33a-9b8955f1ddad None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:50.950544 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d94f1b-6bf4-4d84-b33a-9b8955f1ddad None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:50.957268 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 424/857] 10.222.0.46 () {56 vars in 1135 bytes} [Thu May 26 14:55:50 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:55:50.958426 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10501051-4905-48ca-afba-8da4c3b0e088 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:50.973603 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 434/858] 10.222.0.46 () {56 vars in 1148 bytes} [Thu May 26 14:55:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 14:55:50.975643 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d94f1b-6bf4-4d84-b33a-9b8955f1ddad None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:51.091150 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10501051-4905-48ca-afba-8da4c3b0e088 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:51.096979 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d94f1b-6bf4-4d84-b33a-9b8955f1ddad None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:51.099815 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:51.113964 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:51.133043 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8ca988-1ca6-4ded-b543-fb2691af443c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:51.134290 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8ca988-1ca6-4ded-b543-fb2691af443c None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:51.134782 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8ca988-1ca6-4ded-b543-fb2691af443c None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:51.135304 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8ca988-1ca6-4ded-b543-fb2691af443c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:51.137498 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c11607-f6ef-46da-b11d-76c3a9986b90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:51.138163 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c11607-f6ef-46da-b11d-76c3a9986b90 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:51.138445 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c11607-f6ef-46da-b11d-76c3a9986b90 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:51.138724 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c11607-f6ef-46da-b11d-76c3a9986b90 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:51.139558 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c11607-f6ef-46da-b11d-76c3a9986b90 None admin] RBAC: Adding query filter params (role.id=eb76213820fa4c4e9b0a5d5ac2a08b41, scope.project.id=a17fc8227d4c4da6b2847b350e912ee3, user.id=8028c520d93449a6a087c281280ee13d) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:55:51.139867 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c11607-f6ef-46da-b11d-76c3a9986b90 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:51.141433 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c11607-f6ef-46da-b11d-76c3a9986b90 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:51.147735 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 435/859] 10.222.0.46 () {56 vars in 1374 bytes} [Thu May 26 14:55:50 2022] GET /identity/v3/role_assignments?user.id=8028c520d93449a6a087c281280ee13d&scope.project.id=a17fc8227d4c4da6b2847b350e912ee3&role.id=eb76213820fa4c4e9b0a5d5ac2a08b41 => generated 608 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 14:55:51.219358 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8ca988-1ca6-4ded-b543-fb2691af443c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:51.221242 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8ca988-1ca6-4ded-b543-fb2691af443c None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:51.241615 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028c348f-3d7b-4380-9665-791441c67fad None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:51.241812 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028c348f-3d7b-4380-9665-791441c67fad None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:51.242087 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028c348f-3d7b-4380-9665-791441c67fad None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:51.243770 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 436/860] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 14:55:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 14:55:51.251562 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146109d1-8383-49f3-a817-5ca8da5d2eb8 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:51.251851 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146109d1-8383-49f3-a817-5ca8da5d2eb8 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:51.252131 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146109d1-8383-49f3-a817-5ca8da5d2eb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:51.274132 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-146109d1-8383-49f3-a817-5ca8da5d2eb8 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:55:51.276469 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-146109d1-8383-49f3-a817-5ca8da5d2eb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:55:51.340413 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 425/861] 10.222.0.46 () {58 vars in 1282 bytes} [Thu May 26 14:55:50 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 26 14:55:51.346679 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8ca988-1ca6-4ded-b543-fb2691af443c None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:51.356439 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:51.373409 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 437/862] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 14:55:51 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 14:55:51.395311 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146109d1-8383-49f3-a817-5ca8da5d2eb8 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:51.466209 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8ca988-1ca6-4ded-b543-fb2691af443c None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:51.512275 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc1bc8b-75ea-47ac-b9d2-806012c2a06f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:51.513016 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc1bc8b-75ea-47ac-b9d2-806012c2a06f None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:51.513300 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc1bc8b-75ea-47ac-b9d2-806012c2a06f None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:51.513626 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc1bc8b-75ea-47ac-b9d2-806012c2a06f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:51.525643 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc1bc8b-75ea-47ac-b9d2-806012c2a06f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:51.527107 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc1bc8b-75ea-47ac-b9d2-806012c2a06f None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:51.533168 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146109d1-8383-49f3-a817-5ca8da5d2eb8 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:51.550944 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-fdc1bc8b-75ea-47ac-b9d2-806012c2a06f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 14:55:51.550944 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:55:51.550944 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:55:51.550944 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:55:51.550944 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:55:51.550944 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:55:51.550944 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:55:51.550944 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:55:51.550944 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:55:51.550944 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:55:51.550944 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:55:51.550944 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:55:51.550944 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 14:55:51.550944 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 14:55:51.550944 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 14:55:51.551893 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 14:55:51.551893 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:55:51.551893 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:55:51.551893 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:55:51.551893 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:55:51.551893 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:55:51.551893 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:55:51.551893 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:55:51.551893 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 26 14:55:51.551893 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:55:51.551893 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 26 14:55:51.551893 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:55:51.551893 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:55:51.551893 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:55:51.551893 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 26 14:55:51.551893 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:55:51.551893 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 26 14:55:51.552772 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 14:55:51.552772 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 14:55:51.552772 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 14:55:51.552772 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 26 14:55:51.552772 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 426/863] 10.222.0.46 () {56 vars in 1108 bytes} [Thu May 26 14:55:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:55:51.554135 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:51.558343 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc1bc8b-75ea-47ac-b9d2-806012c2a06f None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:51.579437 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819a7e1c-16db-4299-9514-a244acdc7cdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:51.580571 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819a7e1c-16db-4299-9514-a244acdc7cdc None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:51.581072 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819a7e1c-16db-4299-9514-a244acdc7cdc None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:51.581561 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819a7e1c-16db-4299-9514-a244acdc7cdc None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:51.582388 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819a7e1c-16db-4299-9514-a244acdc7cdc None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:51.583914 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-819a7e1c-16db-4299-9514-a244acdc7cdc None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:51.596938 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-819a7e1c-16db-4299-9514-a244acdc7cdc None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 26 14:55:51.596938 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:55:51.596938 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 14:55:51.596938 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 14:55:51.596938 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 14:55:51.596938 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 14:55:51.596938 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 14:55:51.596938 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 14:55:51.596938 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 26 14:55:51.596938 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 26 14:55:51.596938 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 14:55:51.596938 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 26 14:55:51.596938 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:55:51.596938 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:55:51.596938 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:55:51.596938 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:55:51.597936 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:55:51.597936 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:55:51.597936 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:55:51.597936 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:55:51.597936 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:55:51.597936 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:55:51.597936 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:55:51.597936 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 14:55:51.597936 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 14:55:51.597936 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 14:55:51.597936 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 14:55:51.597936 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:55:51.597936 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:55:51.597936 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:55:51.597936 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:55:51.597936 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:55:51.598773 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:55:51.598773 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:55:51.598773 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 26 14:55:51.598773 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:55:51.598773 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 26 14:55:51.598773 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:55:51.598773 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:55:51.598773 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:55:51.598773 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 26 14:55:51.598773 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:55:51.598773 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 26 14:55:51.598773 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 14:55:51.598773 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 14:55:51.598773 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 26 14:55:51.598773 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 26 14:55:51.606469 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 438/864] 10.222.0.46 () {56 vars in 1132 bytes} [Thu May 26 14:55:51 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 14:55:51.608242 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819a7e1c-16db-4299-9514-a244acdc7cdc None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:51.623762 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:51.680896 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc1bc8b-75ea-47ac-b9d2-806012c2a06f None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:51.724274 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819a7e1c-16db-4299-9514-a244acdc7cdc None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:51.729931 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93da9a4c-e179-4550-948c-8d4d2ae16fea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:51.730711 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93da9a4c-e179-4550-948c-8d4d2ae16fea None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:51.731065 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93da9a4c-e179-4550-948c-8d4d2ae16fea None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:51.731418 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93da9a4c-e179-4550-948c-8d4d2ae16fea None admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:51.732224 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93da9a4c-e179-4550-948c-8d4d2ae16fea None admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:55:51.732603 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93da9a4c-e179-4550-948c-8d4d2ae16fea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:51.734667 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93da9a4c-e179-4550-948c-8d4d2ae16fea None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:51.753523 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 427/865] 10.222.0.46 () {56 vars in 1109 bytes} [Thu May 26 14:55:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:55:51.757196 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93da9a4c-e179-4550-948c-8d4d2ae16fea None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:51.775990 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169619fd-b0a6-4e1c-8270-690eb15f9e96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:51.778099 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169619fd-b0a6-4e1c-8270-690eb15f9e96 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:51.778597 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169619fd-b0a6-4e1c-8270-690eb15f9e96 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:51.779101 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169619fd-b0a6-4e1c-8270-690eb15f9e96 None admin] PATH_INFO: `/v3/services` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:51.780019 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169619fd-b0a6-4e1c-8270-690eb15f9e96 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:55:51.781068 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169619fd-b0a6-4e1c-8270-690eb15f9e96 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:51.782964 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169619fd-b0a6-4e1c-8270-690eb15f9e96 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:51.790880 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 439/866] 10.222.0.46 () {56 vars in 1126 bytes} [Thu May 26 14:55:51 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:55:51.796052 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169619fd-b0a6-4e1c-8270-690eb15f9e96 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:51.847781 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:51.869416 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93da9a4c-e179-4550-948c-8d4d2ae16fea None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:51.887191 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:51.907166 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52249162-03a6-463d-b1df-34860a9c1261 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:51.907966 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52249162-03a6-463d-b1df-34860a9c1261 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:51.908249 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52249162-03a6-463d-b1df-34860a9c1261 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:51.908526 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52249162-03a6-463d-b1df-34860a9c1261 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:51.915848 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52249162-03a6-463d-b1df-34860a9c1261 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:51.917055 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169619fd-b0a6-4e1c-8270-690eb15f9e96 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:51.917277 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52249162-03a6-463d-b1df-34860a9c1261 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:51.932227 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-52249162-03a6-463d-b1df-34860a9c1261 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 14:55:51.932227 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:55:51.932227 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:55:51.932227 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:55:51.932227 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:55:51.932227 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:55:51.932227 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:55:51.932227 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:55:51.932227 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:55:51.932227 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:55:51.932227 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:55:51.932227 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:55:51.932227 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 14:55:51.932227 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 14:55:51.932227 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 14:55:51.934024 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 14:55:51.934024 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:55:51.934024 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:55:51.934024 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:55:51.934024 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:55:51.934024 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:55:51.934024 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:55:51.934024 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:55:51.934024 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 26 14:55:51.934024 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:55:51.934024 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 26 14:55:51.934024 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:55:51.934024 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:55:51.934024 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:55:51.934024 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 26 14:55:51.934024 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:55:51.934024 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 26 14:55:51.934928 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 14:55:51.934928 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 14:55:51.934928 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 14:55:51.934928 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 14:55:51.934928 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 14:55:51.934928 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 14:55:51.934928 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 14:55:51.934928 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 26 14:55:51.934928 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 428/867] 10.222.0.46 () {56 vars in 1111 bytes} [Thu May 26 14:55:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 14:55:51.934928 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3337dbb9-5433-40c2-89e7-9b841586081f None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:51.935475 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3337dbb9-5433-40c2-89e7-9b841586081f None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:51.935475 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3337dbb9-5433-40c2-89e7-9b841586081f None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:51.936605 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 429/868] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 14:55:51 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 14:55:51.941151 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3337dbb9-5433-40c2-89e7-9b841586081f None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:51.958408 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0bb9ac-c804-41b2-8c85-b20816fe820b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:51.959088 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0bb9ac-c804-41b2-8c85-b20816fe820b None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:51.959369 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0bb9ac-c804-41b2-8c85-b20816fe820b None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:51.959646 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0bb9ac-c804-41b2-8c85-b20816fe820b None admin] PATH_INFO: `/v3/services` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:51.960351 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0bb9ac-c804-41b2-8c85-b20816fe820b None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:55:51.960680 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0bb9ac-c804-41b2-8c85-b20816fe820b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:51.962042 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0bb9ac-c804-41b2-8c85-b20816fe820b None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:51.977502 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 440/869] 10.222.0.46 () {56 vars in 1126 bytes} [Thu May 26 14:55:51 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:55:51.977502 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec24d5fa-71fc-4b36-9457-8073bf8b8e86 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:51.977502 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec24d5fa-71fc-4b36-9457-8073bf8b8e86 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:51.977502 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec24d5fa-71fc-4b36-9457-8073bf8b8e86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:52.002144 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-ec24d5fa-71fc-4b36-9457-8073bf8b8e86 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:55:52.003681 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-ec24d5fa-71fc-4b36-9457-8073bf8b8e86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:55:52.087055 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3337dbb9-5433-40c2-89e7-9b841586081f None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:52.097836 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 441/870] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 14:55:51 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 14:55:52.114280 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec24d5fa-71fc-4b36-9457-8073bf8b8e86 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:52.133805 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76274be8-8b30-4298-a408-bc4c61408418 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:52.134636 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76274be8-8b30-4298-a408-bc4c61408418 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:52.134982 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76274be8-8b30-4298-a408-bc4c61408418 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:52.135321 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76274be8-8b30-4298-a408-bc4c61408418 None admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:52.136189 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76274be8-8b30-4298-a408-bc4c61408418 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:55:52.136561 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76274be8-8b30-4298-a408-bc4c61408418 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:52.138180 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76274be8-8b30-4298-a408-bc4c61408418 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:52.152527 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 430/871] 10.222.0.46 () {56 vars in 1148 bytes} [Thu May 26 14:55:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:55:52.153292 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:52.160295 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76274be8-8b30-4298-a408-bc4c61408418 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:52.217084 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:52.253134 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec24d5fa-71fc-4b36-9457-8073bf8b8e86 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:52.277748 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76274be8-8b30-4298-a408-bc4c61408418 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:52.296671 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79c937b-4631-4841-92b2-190af71b59ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:52.297346 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79c937b-4631-4841-92b2-190af71b59ce None admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:52.297656 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79c937b-4631-4841-92b2-190af71b59ce None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:52.298089 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79c937b-4631-4841-92b2-190af71b59ce None admin] PATH_INFO: `/v3/services` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:52.299134 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79c937b-4631-4841-92b2-190af71b59ce None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:52.300511 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79c937b-4631-4841-92b2-190af71b59ce None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:52.322238 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 442/872] 10.222.0.46 () {60 vars in 1141 bytes} [Thu May 26 14:55:52 2022] POST /identity/v3/services => generated 251 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 14:55:52.330096 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1e033b-10e9-4328-a418-d1f0e4121445 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:52.330784 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1e033b-10e9-4328-a418-d1f0e4121445 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:52.331069 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1e033b-10e9-4328-a418-d1f0e4121445 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:52.331401 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1e033b-10e9-4328-a418-d1f0e4121445 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:52.332264 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1e033b-10e9-4328-a418-d1f0e4121445 None admin] RBAC: Adding query filter params (role.id=a2e551773cbd4df084020676865b8222, scope.project.id=a17fc8227d4c4da6b2847b350e912ee3, user.id=a4bea272624b45d8ac2ad27bd07df885) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:55:52.332584 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1e033b-10e9-4328-a418-d1f0e4121445 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:52.334218 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1e033b-10e9-4328-a418-d1f0e4121445 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:52.342030 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 431/873] 10.222.0.46 () {56 vars in 1374 bytes} [Thu May 26 14:55:52 2022] GET /identity/v3/role_assignments?user.id=a4bea272624b45d8ac2ad27bd07df885&scope.project.id=a17fc8227d4c4da6b2847b350e912ee3&role.id=a2e551773cbd4df084020676865b8222 => generated 608 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:55:53.571374 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bae7fc-27b2-446e-a9a8-4c7343ff0e93 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:53.572499 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bae7fc-27b2-446e-a9a8-4c7343ff0e93 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:53.572499 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bae7fc-27b2-446e-a9a8-4c7343ff0e93 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:53.574254 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 443/874] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 14:55:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 14:55:53.579882 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc5c1dc-da22-41d9-a86d-ff5ab7674847 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:53.580337 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc5c1dc-da22-41d9-a86d-ff5ab7674847 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:53.580536 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc5c1dc-da22-41d9-a86d-ff5ab7674847 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:53.601315 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-3cc5c1dc-da22-41d9-a86d-ff5ab7674847 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:55:53.605496 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-3cc5c1dc-da22-41d9-a86d-ff5ab7674847 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:55:53.684694 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 432/875] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 14:55:53 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 14:55:53.708161 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bae7fc-27b2-446e-a9a8-4c7343ff0e93 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:53.802391 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:53.814900 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bae7fc-27b2-446e-a9a8-4c7343ff0e93 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:53.850987 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdc3167-8a13-4253-90b4-2310328d964a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:53.851710 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdc3167-8a13-4253-90b4-2310328d964a None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:53.851991 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdc3167-8a13-4253-90b4-2310328d964a None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:53.852263 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdc3167-8a13-4253-90b4-2310328d964a None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:53.863390 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fdc3167-8a13-4253-90b4-2310328d964a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 14:55:53.872322 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-0fdc3167-8a13-4253-90b4-2310328d964a None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 26 14:55:53.872322 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:55:53.872322 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:55:53.872322 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:55:53.872322 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:55:53.872322 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:55:53.872322 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:55:53.872322 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:55:53.872322 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:55:53.872322 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:55:53.872322 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:55:53.872322 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:55:53.872322 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 14:55:53.872322 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 14:55:53.872322 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 14:55:53.872322 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 14:55:53.873302 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 14:55:53.873302 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 14:55:53.873302 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 14:55:53.873302 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 14:55:53.873302 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:55:53.873302 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:55:53.873302 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:55:53.873302 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:55:53.873302 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:55:53.873302 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:55:53.873302 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:55:53.873302 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 26 14:55:53.873302 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:55:53.873302 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 26 14:55:53.873302 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:55:53.873302 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:55:53.874199 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:55:53.874199 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 26 14:55:53.874199 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:55:53.874199 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 26 14:55:53.874199 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 14:55:53.874199 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 14:55:53.874199 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 14:55:53.874199 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 14:55:53.874199 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 14:55:53.874199 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 14:55:53.874199 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 26 14:55:53.874199 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 26 14:55:53.874199 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 444/876] 10.222.0.46 () {56 vars in 1120 bytes} [Thu May 26 14:55:53 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:55:53.879918 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc5c1dc-da22-41d9-a86d-ff5ab7674847 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:53.913911 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:53.982789 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc5c1dc-da22-41d9-a86d-ff5ab7674847 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:54.024169 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2be549-6930-4bd1-a94b-332d3f9edfaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:54.025923 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2be549-6930-4bd1-a94b-332d3f9edfaf None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:54.026413 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2be549-6930-4bd1-a94b-332d3f9edfaf None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:54.026889 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2be549-6930-4bd1-a94b-332d3f9edfaf None admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:54.027834 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2be549-6930-4bd1-a94b-332d3f9edfaf None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:55:54.028341 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2be549-6930-4bd1-a94b-332d3f9edfaf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:54.029970 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2be549-6930-4bd1-a94b-332d3f9edfaf None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:54.039051 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 433/877] 10.222.0.46 () {56 vars in 1115 bytes} [Thu May 26 14:55:53 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:55:54.046994 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdc3167-8a13-4253-90b4-2310328d964a None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:54.151318 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdc3167-8a13-4253-90b4-2310328d964a None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:54.190269 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a2ffb1-7520-43d8-8686-56401f8042d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:54.191064 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a2ffb1-7520-43d8-8686-56401f8042d1 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:54.191345 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a2ffb1-7520-43d8-8686-56401f8042d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:54.191612 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a2ffb1-7520-43d8-8686-56401f8042d1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:54.203144 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a2ffb1-7520-43d8-8686-56401f8042d1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:54.204600 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a2ffb1-7520-43d8-8686-56401f8042d1 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:54.222849 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-b7a2ffb1-7520-43d8-8686-56401f8042d1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 14:55:54.222849 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:55:54.222849 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:55:54.222849 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:55:54.222849 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:55:54.222849 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:55:54.222849 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:55:54.222849 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:55:54.222849 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:55:54.222849 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:55:54.222849 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:55:54.222849 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:55:54.222849 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 14:55:54.222849 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 14:55:54.222849 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 14:55:54.223871 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 14:55:54.223871 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:55:54.223871 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:55:54.223871 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:55:54.223871 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:55:54.223871 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:55:54.223871 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:55:54.223871 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:55:54.223871 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 26 14:55:54.223871 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:55:54.223871 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 26 14:55:54.223871 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:55:54.223871 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:55:54.223871 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:55:54.223871 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 26 14:55:54.223871 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:55:54.223871 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 26 14:55:54.224788 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 14:55:54.224788 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 14:55:54.224788 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 14:55:54.224788 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 26 14:55:54.224788 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 445/878] 10.222.0.46 () {56 vars in 1108 bytes} [Thu May 26 14:55:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:55:54.229853 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2be549-6930-4bd1-a94b-332d3f9edfaf None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:54.327441 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:54.338765 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2be549-6930-4bd1-a94b-332d3f9edfaf None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:54.359981 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f40e9e-0717-451a-b13a-8629b128ef00 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:54.360294 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f40e9e-0717-451a-b13a-8629b128ef00 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:54.360549 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f40e9e-0717-451a-b13a-8629b128ef00 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:54.362198 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 446/879] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 14:55:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 14:55:54.367346 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff358ea-94e4-420b-95d9-c6920cbef32e None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:54.367636 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff358ea-94e4-420b-95d9-c6920cbef32e None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:54.367916 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff358ea-94e4-420b-95d9-c6920cbef32e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:54.374381 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:54.378761 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b003bd-7d35-4bac-8a22-edb1a09f0264 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:54.379707 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b003bd-7d35-4bac-8a22-edb1a09f0264 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:54.379988 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b003bd-7d35-4bac-8a22-edb1a09f0264 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:54.380262 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b003bd-7d35-4bac-8a22-edb1a09f0264 None admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:54.380986 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b003bd-7d35-4bac-8a22-edb1a09f0264 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:55:54.381289 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b003bd-7d35-4bac-8a22-edb1a09f0264 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:54.383267 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b003bd-7d35-4bac-8a22-edb1a09f0264 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:54.388638 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-fff358ea-94e4-420b-95d9-c6920cbef32e None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:55:54.389873 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-fff358ea-94e4-420b-95d9-c6920cbef32e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:55:54.397853 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 434/880] 10.222.0.46 () {56 vars in 1109 bytes} [Thu May 26 14:55:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:55:54.405205 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b003bd-7d35-4bac-8a22-edb1a09f0264 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:54.466664 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 447/881] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 14:55:54 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 26 14:55:54.491153 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff358ea-94e4-420b-95d9-c6920cbef32e None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:54.524666 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b003bd-7d35-4bac-8a22-edb1a09f0264 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:54.574238 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff911044-918e-4fa9-a06f-4c0f5f34a661 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:54.575855 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff911044-918e-4fa9-a06f-4c0f5f34a661 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:54.576149 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff911044-918e-4fa9-a06f-4c0f5f34a661 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:54.576417 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff911044-918e-4fa9-a06f-4c0f5f34a661 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:54.586314 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff911044-918e-4fa9-a06f-4c0f5f34a661 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:54.587931 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff911044-918e-4fa9-a06f-4c0f5f34a661 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:54.608806 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-ff911044-918e-4fa9-a06f-4c0f5f34a661 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 26 14:55:54.608806 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:55:54.608806 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:55:54.608806 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:55:54.608806 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:55:54.608806 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:55:54.608806 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:55:54.608806 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:55:54.608806 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:55:54.608806 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:55:54.608806 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:55:54.608806 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:55:54.608806 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 14:55:54.608806 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 14:55:54.608806 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 14:55:54.608806 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 14:55:54.614468 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:55:54.614468 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:55:54.614468 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 14:55:54.614468 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 14:55:54.614468 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 14:55:54.614468 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 14:55:54.614468 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:55:54.614468 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:55:54.614468 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:55:54.614468 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:55:54.614468 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:55:54.614468 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 26 14:55:54.614468 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:55:54.614468 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 26 14:55:54.614468 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:55:54.614468 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:55:54.614468 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:55:54.614468 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 26 14:55:54.615466 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:55:54.615466 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 26 14:55:54.615466 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 14:55:54.615466 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 14:55:54.615466 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 14:55:54.615466 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 14:55:54.615466 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 14:55:54.615466 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 14:55:54.615466 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 26 14:55:54.615466 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 26 14:55:54.615466 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 435/882] 10.222.0.46 () {56 vars in 1117 bytes} [Thu May 26 14:55:54 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:55:54.617067 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff358ea-94e4-420b-95d9-c6920cbef32e None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:54.617984 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff911044-918e-4fa9-a06f-4c0f5f34a661 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:54.635515 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:54.657138 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:54.664317 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afe407b-e7f6-4968-a001-a621828a280b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:54.665821 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afe407b-e7f6-4968-a001-a621828a280b None admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:54.666308 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afe407b-e7f6-4968-a001-a621828a280b None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:54.666780 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afe407b-e7f6-4968-a001-a621828a280b None admin] PATH_INFO: `/v3/services` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:54.667726 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afe407b-e7f6-4968-a001-a621828a280b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:54.669627 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afe407b-e7f6-4968-a001-a621828a280b None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:54.689965 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 448/883] 10.222.0.46 () {60 vars in 1141 bytes} [Thu May 26 14:55:54 2022] POST /identity/v3/services => generated 269 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 14:55:54.691264 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cdf49c-44f9-4a0e-9c4c-10e95f89def6 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:54.691552 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cdf49c-44f9-4a0e-9c4c-10e95f89def6 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:54.691830 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cdf49c-44f9-4a0e-9c4c-10e95f89def6 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:54.693282 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 449/884] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 14:55:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 14:55:54.701799 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3db49b-7eea-4b95-a1d8-1ede3e604f20 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:54.701799 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3db49b-7eea-4b95-a1d8-1ede3e604f20 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:54.701799 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3db49b-7eea-4b95-a1d8-1ede3e604f20 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:54.703463 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 450/885] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 14:55:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 14:55:54.705575 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021cc3d5-d2e7-4eb9-9047-1fd53f835392 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:54.705901 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021cc3d5-d2e7-4eb9-9047-1fd53f835392 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:54.706253 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-021cc3d5-d2e7-4eb9-9047-1fd53f835392 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:54.726228 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-021cc3d5-d2e7-4eb9-9047-1fd53f835392 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:55:54.727464 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-021cc3d5-d2e7-4eb9-9047-1fd53f835392 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:55:54.749598 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff911044-918e-4fa9-a06f-4c0f5f34a661 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:54.796452 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013c75b9-07f4-4ca6-b2a5-846f81ac1c3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:54.798025 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013c75b9-07f4-4ca6-b2a5-846f81ac1c3f None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:54.798798 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013c75b9-07f4-4ca6-b2a5-846f81ac1c3f None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:54.799289 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013c75b9-07f4-4ca6-b2a5-846f81ac1c3f None admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:54.800845 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013c75b9-07f4-4ca6-b2a5-846f81ac1c3f None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:55:54.801523 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013c75b9-07f4-4ca6-b2a5-846f81ac1c3f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:54.803437 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013c75b9-07f4-4ca6-b2a5-846f81ac1c3f None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:54.808160 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 451/886] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 14:55:54 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 14:55:54.809587 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631431bb-477b-4d3c-b51d-4239a28f1f8d None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:54.809932 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631431bb-477b-4d3c-b51d-4239a28f1f8d None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:54.810266 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631431bb-477b-4d3c-b51d-4239a28f1f8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:54.827341 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 436/887] 10.222.0.46 () {56 vars in 1149 bytes} [Thu May 26 14:55:54 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:55:54.833293 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013c75b9-07f4-4ca6-b2a5-846f81ac1c3f None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:54.836951 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-631431bb-477b-4d3c-b51d-4239a28f1f8d None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:55:54.839682 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-631431bb-477b-4d3c-b51d-4239a28f1f8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:55:54.908917 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:54.922204 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 452/888] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 14:55:54 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 14:55:54.922977 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631431bb-477b-4d3c-b51d-4239a28f1f8d None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:54.959105 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013c75b9-07f4-4ca6-b2a5-846f81ac1c3f None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:54.982246 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:54.997721 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70af2b5-dbdc-44d7-8cd9-d25f18769e21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:54.998376 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70af2b5-dbdc-44d7-8cd9-d25f18769e21 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:54.998664 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70af2b5-dbdc-44d7-8cd9-d25f18769e21 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:54.998938 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70af2b5-dbdc-44d7-8cd9-d25f18769e21 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:55.035874 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631431bb-477b-4d3c-b51d-4239a28f1f8d None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:55.073295 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70af2b5-dbdc-44d7-8cd9-d25f18769e21 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:55.074533 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d4d6ea-f751-49cd-9c2d-98bb0e44d2d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:55.074809 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70af2b5-dbdc-44d7-8cd9-d25f18769e21 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:55.075390 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d4d6ea-f751-49cd-9c2d-98bb0e44d2d3 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:55.075682 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d4d6ea-f751-49cd-9c2d-98bb0e44d2d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:55.075963 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d4d6ea-f751-49cd-9c2d-98bb0e44d2d3 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:55.076736 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d4d6ea-f751-49cd-9c2d-98bb0e44d2d3 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:55.078139 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d4d6ea-f751-49cd-9c2d-98bb0e44d2d3 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:55.090546 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-77d4d6ea-f751-49cd-9c2d-98bb0e44d2d3 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 26 14:55:55.090546 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:55:55.090546 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 14:55:55.090546 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 14:55:55.090546 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 14:55:55.090546 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 14:55:55.090546 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 14:55:55.090546 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 14:55:55.090546 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 26 14:55:55.090546 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 26 14:55:55.090546 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 14:55:55.090546 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 26 14:55:55.090546 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:55:55.090546 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:55:55.090546 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:55:55.090546 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:55:55.090546 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:55:55.091596 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:55:55.091596 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:55:55.091596 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:55:55.091596 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:55:55.091596 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:55:55.091596 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:55:55.091596 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 14:55:55.091596 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 14:55:55.091596 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 14:55:55.091596 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 14:55:55.091596 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:55:55.091596 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:55:55.091596 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:55:55.091596 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:55:55.091596 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:55:55.091596 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:55:55.092440 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:55:55.092440 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 26 14:55:55.092440 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:55:55.092440 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 26 14:55:55.092440 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:55:55.092440 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:55:55.092440 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:55:55.092440 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 26 14:55:55.092440 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:55:55.092440 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 26 14:55:55.092440 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 14:55:55.092440 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 14:55:55.092440 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 26 14:55:55.092440 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 26 14:55:55.092440 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 453/889] 10.222.0.46 () {56 vars in 1114 bytes} [Thu May 26 14:55:54 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:55:55.093221 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d4d6ea-f751-49cd-9c2d-98bb0e44d2d3 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:55.158053 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:55.200497 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 437/890] 10.222.0.46 () {58 vars in 1282 bytes} [Thu May 26 14:55:54 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 26 14:55:55.201164 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70af2b5-dbdc-44d7-8cd9-d25f18769e21 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:55.216425 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d4d6ea-f751-49cd-9c2d-98bb0e44d2d3 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:55.232999 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:55.252263 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfe1dfa-a5b6-4490-b5d8-d438b7ddabf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:55.253072 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfe1dfa-a5b6-4490-b5d8-d438b7ddabf6 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:55.253352 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfe1dfa-a5b6-4490-b5d8-d438b7ddabf6 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:55.253651 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfe1dfa-a5b6-4490-b5d8-d438b7ddabf6 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:55.254556 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfe1dfa-a5b6-4490-b5d8-d438b7ddabf6 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:55.255911 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfe1dfa-a5b6-4490-b5d8-d438b7ddabf6 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:55.268488 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-9bfe1dfa-a5b6-4490-b5d8-d438b7ddabf6 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 26 14:55:55.268488 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:55:55.268488 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 14:55:55.268488 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 14:55:55.268488 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 14:55:55.268488 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 14:55:55.268488 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 14:55:55.268488 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 14:55:55.268488 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 26 14:55:55.268488 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 26 14:55:55.268488 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 14:55:55.268488 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 26 14:55:55.268488 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:55:55.268488 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:55:55.268488 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:55:55.268488 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:55:55.268488 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:55:55.269719 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:55:55.269719 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:55:55.269719 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:55:55.269719 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:55:55.269719 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:55:55.269719 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:55:55.269719 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 14:55:55.269719 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 14:55:55.269719 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 14:55:55.269719 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 14:55:55.269719 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:55:55.269719 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:55:55.269719 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:55:55.269719 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:55:55.269719 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:55:55.269719 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:55:55.270763 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:55:55.270763 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 26 14:55:55.270763 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:55:55.270763 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 26 14:55:55.270763 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:55:55.270763 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:55:55.270763 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:55:55.270763 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 26 14:55:55.270763 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:55:55.270763 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 26 14:55:55.270763 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 14:55:55.270763 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 14:55:55.270763 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 26 14:55:55.270763 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 26 14:55:55.272914 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 454/891] 10.222.0.46 () {56 vars in 1126 bytes} [Thu May 26 14:55:55 2022] GET /identity/v3/services/object-store => generated 93 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:55:55.273861 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfe1dfa-a5b6-4490-b5d8-d438b7ddabf6 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:55.312616 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70af2b5-dbdc-44d7-8cd9-d25f18769e21 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:55.347167 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b83e09-4167-43f8-a03b-c42d5005acda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:55.347833 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b83e09-4167-43f8-a03b-c42d5005acda None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:55.348117 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b83e09-4167-43f8-a03b-c42d5005acda None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:55.348392 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b83e09-4167-43f8-a03b-c42d5005acda None admin] PATH_INFO: `/v3/services` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:55.349092 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b83e09-4167-43f8-a03b-c42d5005acda None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:55:55.349398 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b83e09-4167-43f8-a03b-c42d5005acda None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:55.351156 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b83e09-4167-43f8-a03b-c42d5005acda None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:55.357812 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 438/892] 10.222.0.46 () {56 vars in 1114 bytes} [Thu May 26 14:55:55 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:55:55.363148 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b83e09-4167-43f8-a03b-c42d5005acda None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:55.386116 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfe1dfa-a5b6-4490-b5d8-d438b7ddabf6 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:55.391285 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:55.424275 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a7c582-48ea-451c-b017-aed3eec8c3dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:55.425305 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a7c582-48ea-451c-b017-aed3eec8c3dd None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:55.425708 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a7c582-48ea-451c-b017-aed3eec8c3dd None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:55.426075 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a7c582-48ea-451c-b017-aed3eec8c3dd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:55.437792 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a7c582-48ea-451c-b017-aed3eec8c3dd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:55.439319 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a7c582-48ea-451c-b017-aed3eec8c3dd None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:55.457208 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-b8a7c582-48ea-451c-b017-aed3eec8c3dd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 14:55:55.457208 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:55:55.457208 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:55:55.457208 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:55:55.457208 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:55:55.457208 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:55:55.457208 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:55:55.457208 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:55:55.457208 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:55:55.457208 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:55:55.457208 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:55:55.457208 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:55:55.457208 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 14:55:55.457208 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 14:55:55.457208 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 14:55:55.458373 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 14:55:55.458373 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:55:55.458373 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:55:55.458373 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:55:55.458373 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:55:55.458373 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:55:55.458373 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:55:55.458373 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:55:55.458373 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 26 14:55:55.458373 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:55:55.458373 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 26 14:55:55.458373 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:55:55.458373 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:55:55.458373 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:55:55.458373 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 26 14:55:55.458373 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:55:55.458373 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 26 14:55:55.459472 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 14:55:55.459472 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 14:55:55.459472 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 14:55:55.459472 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 26 14:55:55.464411 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 455/893] 10.222.0.46 () {56 vars in 1108 bytes} [Thu May 26 14:55:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:55:55.464411 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a7c582-48ea-451c-b017-aed3eec8c3dd None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:55.469950 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b83e09-4167-43f8-a03b-c42d5005acda None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:55.494925 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:55.506502 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb7e0a4-110a-40cc-8732-455a7817d059 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:55.507263 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb7e0a4-110a-40cc-8732-455a7817d059 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:55.507610 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb7e0a4-110a-40cc-8732-455a7817d059 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:55.507954 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb7e0a4-110a-40cc-8732-455a7817d059 None admin] PATH_INFO: `/v3/services` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:55.508731 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb7e0a4-110a-40cc-8732-455a7817d059 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:55:55.509144 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb7e0a4-110a-40cc-8732-455a7817d059 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:55.510609 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb7e0a4-110a-40cc-8732-455a7817d059 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:55.515760 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 439/894] 10.222.0.46 () {56 vars in 1122 bytes} [Thu May 26 14:55:55 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:55:55.516570 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb7e0a4-110a-40cc-8732-455a7817d059 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:55.574437 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a7c582-48ea-451c-b017-aed3eec8c3dd None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:55.611410 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703610f6-3f72-431f-ab8f-5b7e3067dd92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:55.612023 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:55.612462 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703610f6-3f72-431f-ab8f-5b7e3067dd92 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:55.612795 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703610f6-3f72-431f-ab8f-5b7e3067dd92 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:55.613299 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703610f6-3f72-431f-ab8f-5b7e3067dd92 None admin] PATH_INFO: `/v3/services` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:55.613804 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703610f6-3f72-431f-ab8f-5b7e3067dd92 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:55:55.614127 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703610f6-3f72-431f-ab8f-5b7e3067dd92 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:55.615532 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703610f6-3f72-431f-ab8f-5b7e3067dd92 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:55.621263 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 456/895] 10.222.0.46 () {56 vars in 1114 bytes} [Thu May 26 14:55:55 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:55:55.621911 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703610f6-3f72-431f-ab8f-5b7e3067dd92 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:55.631707 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb7e0a4-110a-40cc-8732-455a7817d059 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:55.671995 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fba2eb-9e3f-43c1-a3fa-7e6d182b5f3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:55.672871 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fba2eb-9e3f-43c1-a3fa-7e6d182b5f3d None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:55.673297 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fba2eb-9e3f-43c1-a3fa-7e6d182b5f3d None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:55.673769 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6fba2eb-9e3f-43c1-a3fa-7e6d182b5f3d None admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:55.674480 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6fba2eb-9e3f-43c1-a3fa-7e6d182b5f3d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:55:55.674851 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6fba2eb-9e3f-43c1-a3fa-7e6d182b5f3d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:55.676143 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6fba2eb-9e3f-43c1-a3fa-7e6d182b5f3d None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:55.692737 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 440/896] 10.222.0.46 () {56 vars in 1109 bytes} [Thu May 26 14:55:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:55:55.692737 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fba2eb-9e3f-43c1-a3fa-7e6d182b5f3d None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:55.727332 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:55.736927 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703610f6-3f72-431f-ab8f-5b7e3067dd92 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:55.771491 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5251d7d1-43ae-4680-b45b-a5cf6fbdfb1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:55.772161 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5251d7d1-43ae-4680-b45b-a5cf6fbdfb1b None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:55.772714 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5251d7d1-43ae-4680-b45b-a5cf6fbdfb1b None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:55.772714 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5251d7d1-43ae-4680-b45b-a5cf6fbdfb1b None admin] PATH_INFO: `/v3/services` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:55.773419 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5251d7d1-43ae-4680-b45b-a5cf6fbdfb1b None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:55:55.773777 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5251d7d1-43ae-4680-b45b-a5cf6fbdfb1b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:55.775674 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5251d7d1-43ae-4680-b45b-a5cf6fbdfb1b None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:55.780461 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 457/897] 10.222.0.46 () {56 vars in 1122 bytes} [Thu May 26 14:55:55 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:55:55.791551 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5251d7d1-43ae-4680-b45b-a5cf6fbdfb1b None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:55.798665 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6fba2eb-9e3f-43c1-a3fa-7e6d182b5f3d None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:55.840525 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b96c63-7614-484b-b350-a8ee55d55a3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:55.841376 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b96c63-7614-484b-b350-a8ee55d55a3b None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:55.841763 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b96c63-7614-484b-b350-a8ee55d55a3b None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:55.842113 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b96c63-7614-484b-b350-a8ee55d55a3b None admin] PATH_INFO: `/v3/endpoints` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:55.843001 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b96c63-7614-484b-b350-a8ee55d55a3b None admin] RBAC: Adding query filter params (interface=public, service_id=b7a7447d8c2b4aa5bcb93febd277edcd, region_id=RegionOne) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:55:55.843382 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b96c63-7614-484b-b350-a8ee55d55a3b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:55.844952 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b96c63-7614-484b-b350-a8ee55d55a3b None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:55.859012 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 441/898] 10.222.0.46 () {56 vars in 1251 bytes} [Thu May 26 14:55:55 2022] GET /identity/v3/endpoints?service_id=b7a7447d8c2b4aa5bcb93febd277edcd&interface=public®ion_id=RegionOne => generated 194 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:55:55.860562 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b96c63-7614-484b-b350-a8ee55d55a3b None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:55.887620 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:55.952523 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5251d7d1-43ae-4680-b45b-a5cf6fbdfb1b None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:55.984257 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b96c63-7614-484b-b350-a8ee55d55a3b None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:55.994032 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3d2cfe-a3aa-4417-91d5-5140b2bdb78a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:55.994837 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3d2cfe-a3aa-4417-91d5-5140b2bdb78a None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:55.995118 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3d2cfe-a3aa-4417-91d5-5140b2bdb78a None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:55.995391 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3d2cfe-a3aa-4417-91d5-5140b2bdb78a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:56.003035 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3d2cfe-a3aa-4417-91d5-5140b2bdb78a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:56.004761 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3d2cfe-a3aa-4417-91d5-5140b2bdb78a None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:56.019348 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-dd3d2cfe-a3aa-4417-91d5-5140b2bdb78a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 14:55:56.019348 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:55:56.019348 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:55:56.019348 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:55:56.019348 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:55:56.019348 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:55:56.019348 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:55:56.019348 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:55:56.019348 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:55:56.019348 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:55:56.019348 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:55:56.019348 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:55:56.019348 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 14:55:56.019348 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 14:55:56.019348 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 14:55:56.020343 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 14:55:56.020343 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:55:56.020343 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:55:56.020343 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:55:56.020343 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:55:56.020343 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:55:56.020343 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:55:56.020343 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:55:56.020343 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 26 14:55:56.020343 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:55:56.020343 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 26 14:55:56.020343 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:55:56.020343 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:55:56.020343 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:55:56.020343 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 26 14:55:56.020343 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:55:56.020343 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 26 14:55:56.021266 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 14:55:56.021266 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 14:55:56.021266 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 14:55:56.021266 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 14:55:56.021266 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 14:55:56.021266 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 14:55:56.021266 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 14:55:56.021266 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 26 14:55:56.021266 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 458/899] 10.222.0.46 () {56 vars in 1111 bytes} [Thu May 26 14:55:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:55:56.028670 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3d2cfe-a3aa-4417-91d5-5140b2bdb78a None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:56.029246 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff74f42-8e55-4c9c-aac6-864f4e69e64d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:56.029949 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff74f42-8e55-4c9c-aac6-864f4e69e64d None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:56.030234 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff74f42-8e55-4c9c-aac6-864f4e69e64d None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:56.030505 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff74f42-8e55-4c9c-aac6-864f4e69e64d None admin] PATH_INFO: `/v3/services` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:56.031191 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff74f42-8e55-4c9c-aac6-864f4e69e64d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:56.032564 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff74f42-8e55-4c9c-aac6-864f4e69e64d None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:56.045591 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 442/900] 10.222.0.46 () {56 vars in 1087 bytes} [Thu May 26 14:55:55 2022] GET /identity/v3/services => generated 1268 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 14:55:56.056605 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:56.158971 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3d2cfe-a3aa-4417-91d5-5140b2bdb78a None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:56.198703 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447ba764-02db-45fb-8382-b93faa2ab8af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:56.199384 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447ba764-02db-45fb-8382-b93faa2ab8af None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:56.199671 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447ba764-02db-45fb-8382-b93faa2ab8af None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:56.199950 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447ba764-02db-45fb-8382-b93faa2ab8af None admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:56.200762 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447ba764-02db-45fb-8382-b93faa2ab8af None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:55:56.201075 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447ba764-02db-45fb-8382-b93faa2ab8af None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:56.202569 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447ba764-02db-45fb-8382-b93faa2ab8af None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:56.217078 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 459/901] 10.222.0.46 () {56 vars in 1148 bytes} [Thu May 26 14:55:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 14:55:56.223063 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff74f42-8e55-4c9c-aac6-864f4e69e64d None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:56.315698 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:56.338890 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff74f42-8e55-4c9c-aac6-864f4e69e64d None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:56.374994 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3e3e63-6aac-4eba-914d-7b947bcb5021 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:56.375657 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3e3e63-6aac-4eba-914d-7b947bcb5021 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:56.375942 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3e3e63-6aac-4eba-914d-7b947bcb5021 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:56.376224 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3e3e63-6aac-4eba-914d-7b947bcb5021 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:56.377063 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3e3e63-6aac-4eba-914d-7b947bcb5021 None admin] RBAC: Adding query filter params (role.id=3593816f89a648809d5d7c9160fb87dc, scope.project.id=a17fc8227d4c4da6b2847b350e912ee3, user.id=8028c520d93449a6a087c281280ee13d) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:55:56.377376 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3e3e63-6aac-4eba-914d-7b947bcb5021 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:56.378846 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3e3e63-6aac-4eba-914d-7b947bcb5021 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:56.386166 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 443/902] 10.222.0.46 () {56 vars in 1374 bytes} [Thu May 26 14:55:56 2022] GET /identity/v3/role_assignments?user.id=8028c520d93449a6a087c281280ee13d&scope.project.id=a17fc8227d4c4da6b2847b350e912ee3&role.id=3593816f89a648809d5d7c9160fb87dc => generated 259 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:55:56.983064 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ffa632-3cb2-4fcf-baef-2633453ebe55 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:56.983582 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ffa632-3cb2-4fcf-baef-2633453ebe55 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:56.983582 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ffa632-3cb2-4fcf-baef-2633453ebe55 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:56.985153 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 460/903] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 14:55:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 14:55:56.990769 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d9450f-042a-4c6e-a1e7-4af1022e17f7 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:56.991260 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d9450f-042a-4c6e-a1e7-4af1022e17f7 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:56.991425 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d9450f-042a-4c6e-a1e7-4af1022e17f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:57.009181 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-f3d9450f-042a-4c6e-a1e7-4af1022e17f7 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:55:57.010415 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-f3d9450f-042a-4c6e-a1e7-4af1022e17f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:55:57.085070 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 444/904] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 14:55:56 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 14:55:57.108197 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ffa632-3cb2-4fcf-baef-2633453ebe55 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:57.199215 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:57.222763 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ffa632-3cb2-4fcf-baef-2633453ebe55 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:57.258613 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c781ad0-15f2-453e-ae9c-702045e8e0f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:57.259381 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c781ad0-15f2-453e-ae9c-702045e8e0f6 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:57.259669 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c781ad0-15f2-453e-ae9c-702045e8e0f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:57.259945 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c781ad0-15f2-453e-ae9c-702045e8e0f6 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:57.260602 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c781ad0-15f2-453e-ae9c-702045e8e0f6 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:57.262014 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c781ad0-15f2-453e-ae9c-702045e8e0f6 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:57.275433 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-6c781ad0-15f2-453e-ae9c-702045e8e0f6 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 26 14:55:57.275433 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:55:57.275433 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 14:55:57.275433 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 14:55:57.275433 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 14:55:57.275433 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 14:55:57.275433 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 14:55:57.275433 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 14:55:57.275433 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 26 14:55:57.275433 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 26 14:55:57.275433 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 14:55:57.275433 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 26 14:55:57.275433 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:55:57.275433 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:55:57.275433 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:55:57.275433 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:55:57.276384 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:55:57.276384 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:55:57.276384 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:55:57.276384 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:55:57.276384 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:55:57.276384 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:55:57.276384 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:55:57.276384 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 14:55:57.276384 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 14:55:57.276384 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 14:55:57.276384 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 14:55:57.276384 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:55:57.276384 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:55:57.276384 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:55:57.276384 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:55:57.276384 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:55:57.277230 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:55:57.277230 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:55:57.277230 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 26 14:55:57.277230 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:55:57.277230 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 26 14:55:57.277230 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:55:57.277230 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:55:57.277230 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:55:57.277230 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 26 14:55:57.277230 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:55:57.277230 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 26 14:55:57.277230 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 14:55:57.277230 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 14:55:57.277230 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 26 14:55:57.277230 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 26 14:55:57.277230 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 461/905] 10.222.0.46 () {56 vars in 1132 bytes} [Thu May 26 14:55:57 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:55:57.282971 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d9450f-042a-4c6e-a1e7-4af1022e17f7 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:57.361080 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:57.387822 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d9450f-042a-4c6e-a1e7-4af1022e17f7 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:57.423722 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce5e528-6593-4462-9002-6fe27be2eb61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:57.424395 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce5e528-6593-4462-9002-6fe27be2eb61 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:57.424679 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce5e528-6593-4462-9002-6fe27be2eb61 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:57.424956 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce5e528-6593-4462-9002-6fe27be2eb61 None admin] PATH_INFO: `/v3/services` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:57.425693 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce5e528-6593-4462-9002-6fe27be2eb61 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:55:57.426008 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce5e528-6593-4462-9002-6fe27be2eb61 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:57.427425 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce5e528-6593-4462-9002-6fe27be2eb61 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:57.437366 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 445/906] 10.222.0.46 () {56 vars in 1126 bytes} [Thu May 26 14:55:57 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:55:57.442673 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c781ad0-15f2-453e-ae9c-702045e8e0f6 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:57.559589 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c781ad0-15f2-453e-ae9c-702045e8e0f6 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:57.596679 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff135f97-34ca-4b83-9e18-2251d0841b4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:57.597783 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff135f97-34ca-4b83-9e18-2251d0841b4e None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:57.598266 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff135f97-34ca-4b83-9e18-2251d0841b4e None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:57.598737 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff135f97-34ca-4b83-9e18-2251d0841b4e None admin] PATH_INFO: `/v3/services` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:57.599635 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff135f97-34ca-4b83-9e18-2251d0841b4e None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:55:57.600136 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff135f97-34ca-4b83-9e18-2251d0841b4e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:57.601726 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff135f97-34ca-4b83-9e18-2251d0841b4e None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:57.608205 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 462/907] 10.222.0.46 () {56 vars in 1126 bytes} [Thu May 26 14:55:57 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:55:57.614572 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce5e528-6593-4462-9002-6fe27be2eb61 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:57.726248 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce5e528-6593-4462-9002-6fe27be2eb61 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:57.769128 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd9055e-dbb4-459c-b5df-be75b8f0cb7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:57.769820 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd9055e-dbb4-459c-b5df-be75b8f0cb7b None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:57.770101 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd9055e-dbb4-459c-b5df-be75b8f0cb7b None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:57.770377 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd9055e-dbb4-459c-b5df-be75b8f0cb7b None admin] PATH_INFO: `/v3/endpoints` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:57.771158 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd9055e-dbb4-459c-b5df-be75b8f0cb7b None admin] RBAC: Adding query filter params (interface=public, service_id=f768329f5d3b4eb096d6f690504430c0, region_id=RegionOne) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:55:57.771467 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd9055e-dbb4-459c-b5df-be75b8f0cb7b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:57.772832 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd9055e-dbb4-459c-b5df-be75b8f0cb7b None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:57.779072 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 446/908] 10.222.0.46 () {56 vars in 1251 bytes} [Thu May 26 14:55:57 2022] GET /identity/v3/endpoints?service_id=f768329f5d3b4eb096d6f690504430c0&interface=public®ion_id=RegionOne => generated 194 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:55:57.786565 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff135f97-34ca-4b83-9e18-2251d0841b4e None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:57.798809 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:57.897699 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff135f97-34ca-4b83-9e18-2251d0841b4e None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:57.933366 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21004950-0c98-49b5-8632-7f43a5bf8335 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:57.934083 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21004950-0c98-49b5-8632-7f43a5bf8335 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:57.934368 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21004950-0c98-49b5-8632-7f43a5bf8335 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:57.934645 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21004950-0c98-49b5-8632-7f43a5bf8335 None admin] PATH_INFO: `/v3/services` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:57.935344 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21004950-0c98-49b5-8632-7f43a5bf8335 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:57.936759 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21004950-0c98-49b5-8632-7f43a5bf8335 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:57.944085 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 463/909] 10.222.0.46 () {56 vars in 1087 bytes} [Thu May 26 14:55:57 2022] GET /identity/v3/services => generated 1268 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 14:55:58.138889 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbfb898-ed92-40cd-a954-632ebcf2b8c5 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:58.139400 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbfb898-ed92-40cd-a954-632ebcf2b8c5 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:58.139502 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbfb898-ed92-40cd-a954-632ebcf2b8c5 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:58.140965 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 447/910] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 14:55:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 14:55:58.146590 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848d6fe0-4e4f-4ca6-aa91-03174d4b9b27 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:58.147425 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848d6fe0-4e4f-4ca6-aa91-03174d4b9b27 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:58.147976 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848d6fe0-4e4f-4ca6-aa91-03174d4b9b27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:58.167898 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-848d6fe0-4e4f-4ca6-aa91-03174d4b9b27 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:55:58.168441 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-848d6fe0-4e4f-4ca6-aa91-03174d4b9b27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:55:58.245676 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 464/911] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 14:55:58 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 14:55:58.263565 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbfb898-ed92-40cd-a954-632ebcf2b8c5 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:58.266610 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:58.378330 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbfb898-ed92-40cd-a954-632ebcf2b8c5 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:58.413568 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed72a3dd-ed8f-4aea-b975-4043c099fc78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:58.414164 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed72a3dd-ed8f-4aea-b975-4043c099fc78 None admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:58.414446 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed72a3dd-ed8f-4aea-b975-4043c099fc78 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:58.414729 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed72a3dd-ed8f-4aea-b975-4043c099fc78 None admin] PATH_INFO: `/v3/services` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:58.416126 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed72a3dd-ed8f-4aea-b975-4043c099fc78 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:58.417597 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed72a3dd-ed8f-4aea-b975-4043c099fc78 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:58.438312 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc469f8f-b683-4686-a0c6-27dafce3239d None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:58.438631 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc469f8f-b683-4686-a0c6-27dafce3239d None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:58.438881 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc469f8f-b683-4686-a0c6-27dafce3239d None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:58.440116 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 448/912] 10.222.0.46 () {60 vars in 1141 bytes} [Thu May 26 14:55:58 2022] POST /identity/v3/services => generated 241 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 14:55:58.442802 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 465/913] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 14:55:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 14:55:58.447695 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960a7c6e-147b-454d-961d-2bbc29999c85 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:58.448037 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960a7c6e-147b-454d-961d-2bbc29999c85 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:58.448256 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960a7c6e-147b-454d-961d-2bbc29999c85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:58.466824 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-960a7c6e-147b-454d-961d-2bbc29999c85 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:55:58.468060 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-960a7c6e-147b-454d-961d-2bbc29999c85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:55:58.538090 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:58.557780 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 449/914] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 14:55:58 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 14:55:58.580538 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc469f8f-b683-4686-a0c6-27dafce3239d None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:58.583630 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:58.619273 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc6f164-e45d-4130-a7b3-653a10646667 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:58.619706 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc6f164-e45d-4130-a7b3-653a10646667 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:58.622900 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc6f164-e45d-4130-a7b3-653a10646667 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:58.622900 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 450/915] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 14:55:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 14:55:58.627148 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ad0c79-1ab2-4f48-b77c-0ac69a3f7884 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:58.627438 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ad0c79-1ab2-4f48-b77c-0ac69a3f7884 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:58.627715 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ad0c79-1ab2-4f48-b77c-0ac69a3f7884 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:58.646067 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-35ad0c79-1ab2-4f48-b77c-0ac69a3f7884 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:55:58.647425 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-35ad0c79-1ab2-4f48-b77c-0ac69a3f7884 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:55:58.709672 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc469f8f-b683-4686-a0c6-27dafce3239d None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:58.736539 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 451/916] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 14:55:58 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 26 14:55:58.757352 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125e27c4-a514-4d47-8cd8-847b28ea61aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:58.758251 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125e27c4-a514-4d47-8cd8-847b28ea61aa None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:58.758532 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125e27c4-a514-4d47-8cd8-847b28ea61aa None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:58.758837 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125e27c4-a514-4d47-8cd8-847b28ea61aa None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:58.759555 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125e27c4-a514-4d47-8cd8-847b28ea61aa None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:58.760985 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125e27c4-a514-4d47-8cd8-847b28ea61aa None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:58.763033 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ad0c79-1ab2-4f48-b77c-0ac69a3f7884 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:58.775868 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-125e27c4-a514-4d47-8cd8-847b28ea61aa None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 26 14:55:58.775868 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:55:58.775868 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 14:55:58.775868 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 14:55:58.775868 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 14:55:58.775868 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 14:55:58.775868 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 14:55:58.775868 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 14:55:58.775868 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 26 14:55:58.775868 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 26 14:55:58.775868 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 14:55:58.775868 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 26 14:55:58.775868 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:55:58.775868 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:55:58.775868 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:55:58.775868 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:55:58.775868 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:55:58.776944 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:55:58.776944 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:55:58.776944 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:55:58.776944 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:55:58.776944 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:55:58.776944 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:55:58.776944 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 14:55:58.776944 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 14:55:58.776944 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 14:55:58.776944 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 14:55:58.776944 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:55:58.776944 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:55:58.776944 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:55:58.776944 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:55:58.776944 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:55:58.776944 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:55:58.777834 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:55:58.777834 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 26 14:55:58.777834 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:55:58.777834 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 26 14:55:58.777834 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:55:58.777834 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:55:58.777834 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:55:58.777834 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 26 14:55:58.777834 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:55:58.777834 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 26 14:55:58.777834 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 14:55:58.777834 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 14:55:58.777834 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 26 14:55:58.777834 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 26 14:55:58.777834 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 466/917] 10.222.0.46 () {56 vars in 1114 bytes} [Thu May 26 14:55:58 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:55:58.784011 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125e27c4-a514-4d47-8cd8-847b28ea61aa None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:58.873678 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:58.887623 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ad0c79-1ab2-4f48-b77c-0ac69a3f7884 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:58.913821 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:58.917594 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125e27c4-a514-4d47-8cd8-847b28ea61aa None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:58.937482 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa72501-5a92-443f-8dea-6add5c9c269f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:58.938444 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa72501-5a92-443f-8dea-6add5c9c269f None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:58.938728 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa72501-5a92-443f-8dea-6add5c9c269f None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:58.939004 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa72501-5a92-443f-8dea-6add5c9c269f None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:58.953121 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa72501-5a92-443f-8dea-6add5c9c269f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 14:55:58.954878 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddb57fc-8e0c-4a10-8300-3d68353b1ad7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:58.955703 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddb57fc-8e0c-4a10-8300-3d68353b1ad7 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:58.956128 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddb57fc-8e0c-4a10-8300-3d68353b1ad7 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:58.956544 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eddb57fc-8e0c-4a10-8300-3d68353b1ad7 None admin] PATH_INFO: `/v3/services` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:58.957485 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddb57fc-8e0c-4a10-8300-3d68353b1ad7 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:55:58.957978 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddb57fc-8e0c-4a10-8300-3d68353b1ad7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:58.959469 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eddb57fc-8e0c-4a10-8300-3d68353b1ad7 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:58.962774 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-7aa72501-5a92-443f-8dea-6add5c9c269f None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 26 14:55:58.962774 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:55:58.962774 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:55:58.962774 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:55:58.962774 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:55:58.962774 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:55:58.962774 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:55:58.962774 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:55:58.962774 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:55:58.962774 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:55:58.962774 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:55:58.962774 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:55:58.962774 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 14:55:58.962774 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 14:55:58.962774 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 14:55:58.962774 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 14:55:58.963672 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 14:55:58.963672 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 14:55:58.963672 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 14:55:58.963672 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 14:55:58.963672 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:55:58.963672 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:55:58.963672 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:55:58.963672 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:55:58.963672 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:55:58.963672 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:55:58.963672 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:55:58.963672 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 26 14:55:58.963672 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:55:58.963672 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 26 14:55:58.963672 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:55:58.963672 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:55:58.964562 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:55:58.964562 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 26 14:55:58.964562 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:55:58.964562 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 26 14:55:58.964562 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 14:55:58.964562 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 14:55:58.964562 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 14:55:58.964562 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 14:55:58.964562 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 14:55:58.964562 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 14:55:58.964562 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 26 14:55:58.964562 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 26 14:55:58.964562 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 452/918] 10.222.0.46 () {56 vars in 1120 bytes} [Thu May 26 14:55:58 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 14:55:58.970399 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 467/919] 10.222.0.46 () {56 vars in 1114 bytes} [Thu May 26 14:55:58 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:55:58.974206 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa72501-5a92-443f-8dea-6add5c9c269f None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:58.974612 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddb57fc-8e0c-4a10-8300-3d68353b1ad7 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:59.086837 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eddb57fc-8e0c-4a10-8300-3d68353b1ad7 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:59.106308 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa72501-5a92-443f-8dea-6add5c9c269f None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:59.121912 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:59.130628 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533d5c34-10a4-4ca7-9e90-a4caa1383dac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:59.131547 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533d5c34-10a4-4ca7-9e90-a4caa1383dac None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:59.131838 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533d5c34-10a4-4ca7-9e90-a4caa1383dac None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:59.132117 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533d5c34-10a4-4ca7-9e90-a4caa1383dac None admin] PATH_INFO: `/v3/services` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:59.133373 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533d5c34-10a4-4ca7-9e90-a4caa1383dac None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:55:59.133982 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533d5c34-10a4-4ca7-9e90-a4caa1383dac None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:59.135667 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533d5c34-10a4-4ca7-9e90-a4caa1383dac None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:59.143099 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 468/920] 10.222.0.46 () {56 vars in 1114 bytes} [Thu May 26 14:55:58 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:55:59.150140 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533d5c34-10a4-4ca7-9e90-a4caa1383dac None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:59.151210 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070b503a-4391-4bfe-b240-8e31c980d36b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:59.151890 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070b503a-4391-4bfe-b240-8e31c980d36b None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:59.152169 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070b503a-4391-4bfe-b240-8e31c980d36b None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:59.152442 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070b503a-4391-4bfe-b240-8e31c980d36b None admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:59.153198 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070b503a-4391-4bfe-b240-8e31c980d36b None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:55:59.153531 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070b503a-4391-4bfe-b240-8e31c980d36b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:59.154921 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070b503a-4391-4bfe-b240-8e31c980d36b None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:59.166960 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 453/921] 10.222.0.46 () {56 vars in 1115 bytes} [Thu May 26 14:55:58 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 14:55:59.174530 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070b503a-4391-4bfe-b240-8e31c980d36b None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:59.222440 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:59.285272 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070b503a-4391-4bfe-b240-8e31c980d36b None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:59.286214 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533d5c34-10a4-4ca7-9e90-a4caa1383dac None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:59.326918 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193f16ca-4011-4391-8f47-98f800634571 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:59.327920 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193f16ca-4011-4391-8f47-98f800634571 None admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:59.328353 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193f16ca-4011-4391-8f47-98f800634571 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:59.328800 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193f16ca-4011-4391-8f47-98f800634571 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:59.329815 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cbd651-c8be-49b3-b2e7-c26300a989ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:59.329982 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193f16ca-4011-4391-8f47-98f800634571 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:59.330752 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cbd651-c8be-49b3-b2e7-c26300a989ba None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:59.331106 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cbd651-c8be-49b3-b2e7-c26300a989ba None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:59.331449 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cbd651-c8be-49b3-b2e7-c26300a989ba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:59.332267 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193f16ca-4011-4391-8f47-98f800634571 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:59.352180 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8cbd651-c8be-49b3-b2e7-c26300a989ba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:59.353732 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8cbd651-c8be-49b3-b2e7-c26300a989ba None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:59.369776 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 469/922] 10.222.0.46 () {60 vars in 1144 bytes} [Thu May 26 14:55:59 2022] POST /identity/v3/endpoints => generated 346 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 14:55:59.380185 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-a8cbd651-c8be-49b3-b2e7-c26300a989ba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 14:55:59.380185 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:55:59.380185 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:55:59.380185 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:55:59.380185 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:55:59.380185 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:55:59.380185 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:55:59.380185 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:55:59.380185 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:55:59.380185 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:55:59.380185 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:55:59.380185 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:55:59.380185 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 14:55:59.380185 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 14:55:59.380185 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 14:55:59.381158 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 14:55:59.381158 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:55:59.381158 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:55:59.381158 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:55:59.381158 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:55:59.381158 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:55:59.381158 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:55:59.381158 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:55:59.381158 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 26 14:55:59.381158 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:55:59.381158 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 26 14:55:59.381158 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:55:59.381158 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:55:59.381158 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:55:59.381158 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 26 14:55:59.381158 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:55:59.381158 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 26 14:55:59.382329 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 14:55:59.382329 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 14:55:59.382329 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 14:55:59.382329 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 26 14:55:59.382329 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 454/923] 10.222.0.46 () {56 vars in 1108 bytes} [Thu May 26 14:55:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:55:59.387764 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193f16ca-4011-4391-8f47-98f800634571 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:59.442199 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:59.505523 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193f16ca-4011-4391-8f47-98f800634571 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:59.542467 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc8d455-0153-4fff-a7df-24dcdf43170c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:59.543579 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc8d455-0153-4fff-a7df-24dcdf43170c None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:59.544078 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc8d455-0153-4fff-a7df-24dcdf43170c None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:59.544562 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc8d455-0153-4fff-a7df-24dcdf43170c None admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:59.545529 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc8d455-0153-4fff-a7df-24dcdf43170c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:55:59.546051 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc8d455-0153-4fff-a7df-24dcdf43170c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:59.547791 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc8d455-0153-4fff-a7df-24dcdf43170c None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:59.562284 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 470/924] 10.222.0.46 () {56 vars in 1109 bytes} [Thu May 26 14:55:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:55:59.568393 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cbd651-c8be-49b3-b2e7-c26300a989ba None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:59.672761 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:59.683239 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cbd651-c8be-49b3-b2e7-c26300a989ba None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:59.723759 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd4d374-c9ba-449a-a195-3e28c8e3ccc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:59.724584 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd4d374-c9ba-449a-a195-3e28c8e3ccc0 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:59.724863 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd4d374-c9ba-449a-a195-3e28c8e3ccc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:59.725145 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd4d374-c9ba-449a-a195-3e28c8e3ccc0 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:59.733766 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd4d374-c9ba-449a-a195-3e28c8e3ccc0 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:59.735230 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd4d374-c9ba-449a-a195-3e28c8e3ccc0 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:59.750839 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-cfd4d374-c9ba-449a-a195-3e28c8e3ccc0 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 26 14:55:59.750839 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:55:59.750839 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:55:59.750839 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:55:59.750839 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:55:59.750839 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:55:59.750839 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:55:59.750839 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:55:59.750839 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:55:59.750839 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:55:59.750839 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:55:59.750839 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:55:59.750839 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 14:55:59.750839 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 14:55:59.750839 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 14:55:59.750839 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 14:55:59.751834 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:55:59.751834 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:55:59.751834 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 14:55:59.751834 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 14:55:59.751834 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 14:55:59.751834 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 14:55:59.751834 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:55:59.751834 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:55:59.751834 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:55:59.751834 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:55:59.751834 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:55:59.751834 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 26 14:55:59.751834 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:55:59.751834 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 26 14:55:59.751834 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:55:59.751834 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:55:59.751834 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:55:59.751834 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 26 14:55:59.752766 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:55:59.752766 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 26 14:55:59.752766 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 14:55:59.752766 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 14:55:59.752766 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 14:55:59.752766 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 14:55:59.752766 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 14:55:59.752766 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 14:55:59.752766 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 26 14:55:59.752766 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 26 14:55:59.752766 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 455/925] 10.222.0.46 () {56 vars in 1117 bytes} [Thu May 26 14:55:59 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:55:59.758465 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc8d455-0153-4fff-a7df-24dcdf43170c None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:55:59.863406 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc8d455-0153-4fff-a7df-24dcdf43170c None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:55:59.873826 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:55:59.906976 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3216d0ab-e657-433c-b84a-d50232155f48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:55:59.907712 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3216d0ab-e657-433c-b84a-d50232155f48 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:55:59.907995 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3216d0ab-e657-433c-b84a-d50232155f48 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:55:59.908309 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3216d0ab-e657-433c-b84a-d50232155f48 None admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:55:59.909154 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3216d0ab-e657-433c-b84a-d50232155f48 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:55:59.909477 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3216d0ab-e657-433c-b84a-d50232155f48 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:55:59.910930 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3216d0ab-e657-433c-b84a-d50232155f48 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:55:59.929828 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 471/926] 10.222.0.46 () {56 vars in 1149 bytes} [Thu May 26 14:55:59 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:55:59.935577 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd4d374-c9ba-449a-a195-3e28c8e3ccc0 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:56:00.041789 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd4d374-c9ba-449a-a195-3e28c8e3ccc0 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:56:00.076730 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6fba10-3726-42b9-8cc3-8024bd0ab98f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:56:00.077557 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6fba10-3726-42b9-8cc3-8024bd0ab98f None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:00.077841 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6fba10-3726-42b9-8cc3-8024bd0ab98f None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:00.078119 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6fba10-3726-42b9-8cc3-8024bd0ab98f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:00.090679 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6fba10-3726-42b9-8cc3-8024bd0ab98f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:56:00.092173 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6fba10-3726-42b9-8cc3-8024bd0ab98f None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:56:00.111891 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-ee6fba10-3726-42b9-8cc3-8024bd0ab98f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 14:56:00.111891 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:56:00.111891 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:56:00.111891 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:56:00.111891 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:56:00.111891 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:56:00.111891 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:56:00.111891 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:56:00.111891 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:56:00.111891 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:56:00.111891 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:56:00.111891 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:56:00.111891 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 14:56:00.111891 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 14:56:00.111891 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 14:56:00.114047 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 14:56:00.114047 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:56:00.114047 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:56:00.114047 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:56:00.114047 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:56:00.114047 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:56:00.114047 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:56:00.114047 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:56:00.114047 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 26 14:56:00.114047 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:56:00.114047 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 26 14:56:00.114047 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:56:00.114047 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:56:00.114047 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:56:00.114047 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 26 14:56:00.114047 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:56:00.114047 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 26 14:56:00.116171 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 14:56:00.116171 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 14:56:00.116171 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 14:56:00.116171 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 26 14:56:00.121473 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 456/927] 10.222.0.46 () {56 vars in 1108 bytes} [Thu May 26 14:55:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:56:00.121788 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3216d0ab-e657-433c-b84a-d50232155f48 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:56:00.223888 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3216d0ab-e657-433c-b84a-d50232155f48 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:56:00.260260 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926dc5eb-2180-49d4-bd58-597eb6a31ff6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:56:00.260929 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926dc5eb-2180-49d4-bd58-597eb6a31ff6 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:00.261209 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926dc5eb-2180-49d4-bd58-597eb6a31ff6 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:00.261581 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926dc5eb-2180-49d4-bd58-597eb6a31ff6 None admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:00.262310 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926dc5eb-2180-49d4-bd58-597eb6a31ff6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:56:00.262613 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926dc5eb-2180-49d4-bd58-597eb6a31ff6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:56:00.263973 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926dc5eb-2180-49d4-bd58-597eb6a31ff6 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:56:00.276280 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 472/928] 10.222.0.46 () {56 vars in 1109 bytes} [Thu May 26 14:56:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:56:00.281561 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6fba10-3726-42b9-8cc3-8024bd0ab98f None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:56:00.285615 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:56:00.294892 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326135cb-f397-4890-b4bf-f609beeecce6 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:00.295249 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326135cb-f397-4890-b4bf-f609beeecce6 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:00.295544 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326135cb-f397-4890-b4bf-f609beeecce6 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:00.297828 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 473/929] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 14:56:00 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 14:56:00.303925 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feaab9c2-69a5-4259-8600-26791aa39838 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:00.304224 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feaab9c2-69a5-4259-8600-26791aa39838 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:00.304510 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feaab9c2-69a5-4259-8600-26791aa39838 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:00.322468 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-feaab9c2-69a5-4259-8600-26791aa39838 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:56:00.323674 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-feaab9c2-69a5-4259-8600-26791aa39838 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:56:00.343191 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:56:00.401105 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6fba10-3726-42b9-8cc3-8024bd0ab98f None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:56:00.421593 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 474/930] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 14:56:00 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 26 14:56:00.441675 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485d777a-c635-4d62-96e0-e77d87eafe70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:56:00.442475 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485d777a-c635-4d62-96e0-e77d87eafe70 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:00.442794 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485d777a-c635-4d62-96e0-e77d87eafe70 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:00.443069 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485d777a-c635-4d62-96e0-e77d87eafe70 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:00.444864 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feaab9c2-69a5-4259-8600-26791aa39838 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:56:00.453944 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485d777a-c635-4d62-96e0-e77d87eafe70 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:56:00.455697 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485d777a-c635-4d62-96e0-e77d87eafe70 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:56:00.470680 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-485d777a-c635-4d62-96e0-e77d87eafe70 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 14:56:00.470680 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:56:00.470680 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:56:00.470680 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:56:00.470680 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:56:00.470680 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:56:00.470680 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:56:00.470680 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:56:00.470680 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:56:00.470680 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:56:00.470680 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:56:00.470680 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:56:00.470680 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 14:56:00.470680 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 14:56:00.470680 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 14:56:00.471911 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 14:56:00.471911 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:56:00.471911 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:56:00.471911 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:56:00.471911 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:56:00.471911 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:56:00.471911 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:56:00.471911 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:56:00.471911 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 26 14:56:00.471911 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:56:00.471911 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 26 14:56:00.471911 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:56:00.471911 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:56:00.471911 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:56:00.471911 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 26 14:56:00.471911 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:56:00.471911 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 26 14:56:00.473133 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 14:56:00.473133 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 14:56:00.473133 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 14:56:00.473133 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 14:56:00.473133 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 14:56:00.473133 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 14:56:00.473133 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 14:56:00.473133 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 26 14:56:00.475354 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 457/931] 10.222.0.46 () {56 vars in 1111 bytes} [Thu May 26 14:56:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:56:00.481445 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485d777a-c635-4d62-96e0-e77d87eafe70 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:56:00.548570 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:56:00.557338 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feaab9c2-69a5-4259-8600-26791aa39838 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:56:00.590624 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485d777a-c635-4d62-96e0-e77d87eafe70 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:56:00.596576 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77605bb6-bd3f-4939-b615-55b620cbab3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:56:00.597387 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77605bb6-bd3f-4939-b615-55b620cbab3d None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:00.597781 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77605bb6-bd3f-4939-b615-55b620cbab3d None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:00.598062 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77605bb6-bd3f-4939-b615-55b620cbab3d None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:00.598709 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77605bb6-bd3f-4939-b615-55b620cbab3d None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:56:00.600048 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77605bb6-bd3f-4939-b615-55b620cbab3d None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:56:00.617377 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-77605bb6-bd3f-4939-b615-55b620cbab3d None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 26 14:56:00.617377 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:56:00.617377 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 14:56:00.617377 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 14:56:00.617377 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 14:56:00.617377 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 14:56:00.617377 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 14:56:00.617377 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 14:56:00.617377 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 26 14:56:00.617377 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 26 14:56:00.617377 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 14:56:00.617377 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 26 14:56:00.617377 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:56:00.617377 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:56:00.617377 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:56:00.617377 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:56:00.619009 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:56:00.619009 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:56:00.619009 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:56:00.619009 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:56:00.619009 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:56:00.619009 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:56:00.619009 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:56:00.619009 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 14:56:00.619009 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 14:56:00.619009 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 14:56:00.619009 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 14:56:00.619009 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:56:00.619009 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:56:00.619009 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:56:00.619009 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:56:00.619009 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:56:00.619888 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:56:00.619888 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:56:00.619888 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 26 14:56:00.619888 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:56:00.619888 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 26 14:56:00.619888 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:56:00.619888 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:56:00.619888 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:56:00.619888 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 26 14:56:00.619888 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:56:00.619888 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 26 14:56:00.619888 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 14:56:00.619888 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 14:56:00.619888 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 26 14:56:00.619888 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 26 14:56:00.624515 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 475/932] 10.222.0.46 () {56 vars in 1132 bytes} [Thu May 26 14:56:00 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 14:56:00.625556 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77605bb6-bd3f-4939-b615-55b620cbab3d None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:56:00.637431 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a976b87b-7033-4b48-b0e4-7251c3bb67f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:56:00.638284 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a976b87b-7033-4b48-b0e4-7251c3bb67f2 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:00.638673 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a976b87b-7033-4b48-b0e4-7251c3bb67f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:00.639050 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a976b87b-7033-4b48-b0e4-7251c3bb67f2 None admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:00.639943 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a976b87b-7033-4b48-b0e4-7251c3bb67f2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:56:00.640357 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a976b87b-7033-4b48-b0e4-7251c3bb67f2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:56:00.641947 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a976b87b-7033-4b48-b0e4-7251c3bb67f2 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:56:00.647943 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:56:00.656366 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 458/933] 10.222.0.46 () {56 vars in 1148 bytes} [Thu May 26 14:56:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 14:56:00.662147 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a976b87b-7033-4b48-b0e4-7251c3bb67f2 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:56:00.750637 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77605bb6-bd3f-4939-b615-55b620cbab3d None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:56:00.779150 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a976b87b-7033-4b48-b0e4-7251c3bb67f2 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:56:00.795766 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0becbb-4d2d-4a94-9662-df9f99322d2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:56:00.796576 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0becbb-4d2d-4a94-9662-df9f99322d2b None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:00.797228 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0becbb-4d2d-4a94-9662-df9f99322d2b None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:00.798102 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0becbb-4d2d-4a94-9662-df9f99322d2b None admin] PATH_INFO: `/v3/services` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:00.799258 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0becbb-4d2d-4a94-9662-df9f99322d2b None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:56:00.799622 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0becbb-4d2d-4a94-9662-df9f99322d2b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:56:00.801801 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0becbb-4d2d-4a94-9662-df9f99322d2b None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:56:00.809347 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:56:00.810279 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 476/934] 10.222.0.46 () {56 vars in 1126 bytes} [Thu May 26 14:56:00 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:56:00.815231 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0becbb-4d2d-4a94-9662-df9f99322d2b None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:56:00.825940 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ee68bd-ca6c-4c61-99c7-d670c1d0bb43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:56:00.826980 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ee68bd-ca6c-4c61-99c7-d670c1d0bb43 None admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:00.827263 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ee68bd-ca6c-4c61-99c7-d670c1d0bb43 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:00.827542 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ee68bd-ca6c-4c61-99c7-d670c1d0bb43 None admin] PATH_INFO: `/v3/projects/a17fc8227d4c4da6b2847b350e912ee3/users/8028c520d93449a6a087c281280ee13d/roles/3593816f89a648809d5d7c9160fb87dc` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:00.830410 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ee68bd-ca6c-4c61-99c7-d670c1d0bb43 None admin] RBAC: Authorizing `identity:create_grant(project_id=a17fc8227d4c4da6b2847b350e912ee3, user_id=8028c520d93449a6a087c281280ee13d, role_id=3593816f89a648809d5d7c9160fb87dc)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:56:00.832789 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ee68bd-ca6c-4c61-99c7-d670c1d0bb43 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:56:00.857554 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 459/935] 10.222.0.46 () {58 vars in 1439 bytes} [Thu May 26 14:56:00 2022] PUT /identity/v3/projects/a17fc8227d4c4da6b2847b350e912ee3/users/8028c520d93449a6a087c281280ee13d/roles/3593816f89a648809d5d7c9160fb87dc => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 14:56:00.859123 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8cc363-588f-4bb0-ad36-251f847976fc None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:00.859417 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8cc363-588f-4bb0-ad36-251f847976fc None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:00.859696 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8cc363-588f-4bb0-ad36-251f847976fc None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:00.861491 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 460/936] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 14:56:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 14:56:00.867417 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fab71c-ff15-4d59-a86c-e825fe4261d3 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:00.867751 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fab71c-ff15-4d59-a86c-e825fe4261d3 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:00.867979 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fab71c-ff15-4d59-a86c-e825fe4261d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:00.886498 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-18fab71c-ff15-4d59-a86c-e825fe4261d3 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:56:00.887724 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-18fab71c-ff15-4d59-a86c-e825fe4261d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:56:00.923009 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:56:00.946547 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0becbb-4d2d-4a94-9662-df9f99322d2b None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:56:00.982904 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 461/937] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 14:56:00 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 26 14:56:00.989696 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0b5743-59d5-4295-8d93-f931730c3972 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:56:00.997521 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0b5743-59d5-4295-8d93-f931730c3972 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:00.997521 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0b5743-59d5-4295-8d93-f931730c3972 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:00.997521 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0b5743-59d5-4295-8d93-f931730c3972 None admin] PATH_INFO: `/v3/services` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:00.997521 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0b5743-59d5-4295-8d93-f931730c3972 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:56:00.997521 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0b5743-59d5-4295-8d93-f931730c3972 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:56:00.997521 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f0b5743-59d5-4295-8d93-f931730c3972 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:56:01.000608 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 477/938] 10.222.0.46 () {56 vars in 1126 bytes} [Thu May 26 14:56:00 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 14:56:01.005936 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fab71c-ff15-4d59-a86c-e825fe4261d3 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:56:01.014423 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0b5743-59d5-4295-8d93-f931730c3972 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:56:01.134335 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18fab71c-ff15-4d59-a86c-e825fe4261d3 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:56:01.146066 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f0b5743-59d5-4295-8d93-f931730c3972 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:56:01.149017 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:56:01.172835 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fb12d8-72dd-49a5-8b7c-66e5674a2911 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:56:01.173764 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fb12d8-72dd-49a5-8b7c-66e5674a2911 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:01.174099 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fb12d8-72dd-49a5-8b7c-66e5674a2911 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:01.174440 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fb12d8-72dd-49a5-8b7c-66e5674a2911 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:01.175105 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42fb12d8-72dd-49a5-8b7c-66e5674a2911 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:56:01.176493 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42fb12d8-72dd-49a5-8b7c-66e5674a2911 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:56:01.187240 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ebc2c8-5441-48d7-9016-a2cb0703330a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:56:01.187898 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ebc2c8-5441-48d7-9016-a2cb0703330a None admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:01.188275 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ebc2c8-5441-48d7-9016-a2cb0703330a None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:01.188594 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ebc2c8-5441-48d7-9016-a2cb0703330a None admin] PATH_INFO: `/v3/endpoints` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:01.189388 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ebc2c8-5441-48d7-9016-a2cb0703330a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:56:01.189883 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-42fb12d8-72dd-49a5-8b7c-66e5674a2911 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 26 14:56:01.189883 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:56:01.189883 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 14:56:01.189883 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 14:56:01.189883 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 14:56:01.189883 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 14:56:01.189883 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 14:56:01.189883 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 14:56:01.189883 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 26 14:56:01.189883 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 26 14:56:01.189883 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 14:56:01.189883 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 26 14:56:01.189883 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:56:01.189883 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:56:01.189883 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:56:01.189883 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:56:01.189883 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:56:01.190856 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:56:01.190856 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:56:01.190856 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:56:01.190856 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:56:01.190856 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:56:01.190856 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:56:01.190856 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 14:56:01.190856 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 14:56:01.190856 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 14:56:01.190856 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 14:56:01.190856 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:56:01.190856 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:56:01.190856 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:56:01.190856 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:56:01.190856 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:56:01.190856 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:56:01.192104 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:56:01.192104 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 26 14:56:01.192104 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:56:01.192104 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 26 14:56:01.192104 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:56:01.192104 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:56:01.192104 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:56:01.192104 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 26 14:56:01.192104 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:56:01.192104 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 26 14:56:01.192104 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 14:56:01.192104 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 14:56:01.192104 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 26 14:56:01.192104 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 26 14:56:01.192859 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ebc2c8-5441-48d7-9016-a2cb0703330a None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:56:01.192932 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 462/939] 10.222.0.46 () {56 vars in 1126 bytes} [Thu May 26 14:56:01 2022] GET /identity/v3/services/object-store => generated 93 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:56:01.201510 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fb12d8-72dd-49a5-8b7c-66e5674a2911 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:56:01.214610 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 478/940] 10.222.0.46 () {60 vars in 1144 bytes} [Thu May 26 14:56:01 2022] POST /identity/v3/endpoints => generated 347 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 14:56:01.328397 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fb12d8-72dd-49a5-8b7c-66e5674a2911 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:56:01.373402 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcc6230-135e-460c-8420-91f8db2072d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:56:01.374188 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcc6230-135e-460c-8420-91f8db2072d8 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:01.374530 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcc6230-135e-460c-8420-91f8db2072d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:01.374816 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcc6230-135e-460c-8420-91f8db2072d8 None admin] PATH_INFO: `/v3/services` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:01.375539 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bcc6230-135e-460c-8420-91f8db2072d8 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:56:01.375849 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bcc6230-135e-460c-8420-91f8db2072d8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:56:01.377243 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bcc6230-135e-460c-8420-91f8db2072d8 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:56:01.383383 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 463/941] 10.222.0.46 () {56 vars in 1122 bytes} [Thu May 26 14:56:01 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:56:01.389970 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ebc2c8-5441-48d7-9016-a2cb0703330a None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:56:01.393569 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:56:01.504430 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ebc2c8-5441-48d7-9016-a2cb0703330a None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:56:01.540510 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682c7ff4-8594-46f5-9dbc-7e8a215776dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:56:01.541559 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682c7ff4-8594-46f5-9dbc-7e8a215776dd None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:01.542049 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682c7ff4-8594-46f5-9dbc-7e8a215776dd None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:01.542566 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682c7ff4-8594-46f5-9dbc-7e8a215776dd None admin] PATH_INFO: `/v3/services` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:01.543471 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682c7ff4-8594-46f5-9dbc-7e8a215776dd None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:56:01.543976 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682c7ff4-8594-46f5-9dbc-7e8a215776dd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:56:01.545564 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682c7ff4-8594-46f5-9dbc-7e8a215776dd None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:56:01.556095 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 479/942] 10.222.0.46 () {56 vars in 1122 bytes} [Thu May 26 14:56:01 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:56:01.557792 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcc6230-135e-460c-8420-91f8db2072d8 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:56:01.608001 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:56:01.680388 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcc6230-135e-460c-8420-91f8db2072d8 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:56:01.734159 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0418c0de-9865-4480-9e2b-3473a9c15a16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:56:01.735268 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0418c0de-9865-4480-9e2b-3473a9c15a16 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:01.735782 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0418c0de-9865-4480-9e2b-3473a9c15a16 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:01.736276 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0418c0de-9865-4480-9e2b-3473a9c15a16 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:01.737805 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0418c0de-9865-4480-9e2b-3473a9c15a16 None admin] RBAC: Adding query filter params (interface=public, service_id=dcb32f2292d8496aa6b34425ff9f2e93, region_id=RegionOne) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:56:01.738379 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0418c0de-9865-4480-9e2b-3473a9c15a16 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:56:01.740855 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0418c0de-9865-4480-9e2b-3473a9c15a16 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:56:01.752638 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 464/943] 10.222.0.46 () {56 vars in 1251 bytes} [Thu May 26 14:56:01 2022] GET /identity/v3/endpoints?service_id=dcb32f2292d8496aa6b34425ff9f2e93&interface=public®ion_id=RegionOne => generated 194 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:56:01.758649 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682c7ff4-8594-46f5-9dbc-7e8a215776dd None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:56:01.831590 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:56:01.854979 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5c2126-b4c5-45c1-bffe-b07bebd00e1d None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:01.855323 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5c2126-b4c5-45c1-bffe-b07bebd00e1d None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:01.855665 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5c2126-b4c5-45c1-bffe-b07bebd00e1d None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:01.857165 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 465/944] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 14:56:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 14:56:01.866065 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e666b9-9fad-4f87-90e1-dee0605ca9b2 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:01.867069 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e666b9-9fad-4f87-90e1-dee0605ca9b2 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:01.867683 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e666b9-9fad-4f87-90e1-dee0605ca9b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:01.872318 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682c7ff4-8594-46f5-9dbc-7e8a215776dd None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:56:01.887632 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-80e666b9-9fad-4f87-90e1-dee0605ca9b2 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:56:01.889158 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-80e666b9-9fad-4f87-90e1-dee0605ca9b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:56:01.911511 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294bd005-c479-4199-9d29-5896db61d3fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:56:01.912576 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294bd005-c479-4199-9d29-5896db61d3fc None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:01.913078 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294bd005-c479-4199-9d29-5896db61d3fc None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:01.914031 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294bd005-c479-4199-9d29-5896db61d3fc None admin] PATH_INFO: `/v3/services` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:01.915253 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294bd005-c479-4199-9d29-5896db61d3fc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:56:01.917302 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294bd005-c479-4199-9d29-5896db61d3fc None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:56:01.926223 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 480/945] 10.222.0.46 () {56 vars in 1087 bytes} [Thu May 26 14:56:01 2022] GET /identity/v3/services => generated 1497 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 14:56:01.974545 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 466/946] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 14:56:01 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 26 14:56:02.000699 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294bd005-c479-4199-9d29-5896db61d3fc None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:56:02.127351 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294bd005-c479-4199-9d29-5896db61d3fc None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:56:02.162588 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:56:02.170390 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aed4d26-b501-4315-ae75-0fa9f7580954 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:56:02.171063 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aed4d26-b501-4315-ae75-0fa9f7580954 None admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:02.171350 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aed4d26-b501-4315-ae75-0fa9f7580954 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:02.171628 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aed4d26-b501-4315-ae75-0fa9f7580954 None admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:02.172340 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aed4d26-b501-4315-ae75-0fa9f7580954 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:56:02.173815 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aed4d26-b501-4315-ae75-0fa9f7580954 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:56:02.175205 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: WARNING keystone.server.flask.common [None req-5aed4d26-b501-4315-ae75-0fa9f7580954 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. May 26 14:56:02.195081 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.sql.core [None req-5aed4d26-b501-4315-ae75-0fa9f7580954 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 26 14:56:02.195081 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] May 26 14:56:02.195081 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [parameters: {'id': 'cc74a3e43aef4c6a8b00134f4bdff0e1', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] May 26 14:56:02.195081 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80168) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 26 14:56:02.203578 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-5aed4d26-b501-4315-ae75-0fa9f7580954 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id cc74a3e43aef4c6a8b00134f4bdff0e1.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id cc74a3e43aef4c6a8b00134f4bdff0e1. May 26 14:56:02.203578 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:56:02.203578 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 26 14:56:02.203578 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application self.dialect.do_execute( May 26 14:56:02.203578 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 26 14:56:02.203578 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 26 14:56:02.203578 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 26 14:56:02.203578 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application result = self._query(query) May 26 14:56:02.203578 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 26 14:56:02.203578 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application conn.query(q) May 26 14:56:02.203578 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 26 14:56:02.203578 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 26 14:56:02.203578 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 26 14:56:02.204428 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application result.read() May 26 14:56:02.204428 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 26 14:56:02.204428 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 26 14:56:02.204428 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 26 14:56:02.204428 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application packet.raise_for_error() May 26 14:56:02.204428 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 26 14:56:02.204428 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 26 14:56:02.204428 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 26 14:56:02.204428 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 26 14:56:02.204428 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 26 14:56:02.204428 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 26 14:56:02.204428 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 26 14:56:02.204428 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 26 14:56:02.204428 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:56:02.204428 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 26 14:56:02.204428 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 26 14:56:02.204428 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project May 26 14:56:02.204428 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return project_ref.to_dict() May 26 14:56:02.205370 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 26 14:56:02.205370 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application next(self.gen) May 26 14:56:02.205370 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 26 14:56:02.205370 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application yield resource May 26 14:56:02.205370 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 26 14:56:02.205370 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application next(self.gen) May 26 14:56:02.205370 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 26 14:56:02.205370 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application self.session.rollback() May 26 14:56:02.205370 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 26 14:56:02.205370 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application self.force_reraise() May 26 14:56:02.205370 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 26 14:56:02.205370 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise self.value May 26 14:56:02.205370 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 26 14:56:02.205370 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 26 14:56:02.205370 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 26 14:56:02.205370 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application session.commit() May 26 14:56:02.205370 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 26 14:56:02.206293 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 26 14:56:02.206293 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 26 14:56:02.206293 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application self._prepare_impl() May 26 14:56:02.206293 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 26 14:56:02.206293 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application self.session.flush() May 26 14:56:02.206293 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 26 14:56:02.206293 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application self._flush(objects) May 26 14:56:02.206293 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 26 14:56:02.206293 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 26 14:56:02.206293 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 26 14:56:02.206293 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application compat.raise_( May 26 14:56:02.206293 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 26 14:56:02.206293 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception May 26 14:56:02.206293 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 26 14:56:02.206293 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application flush_context.execute() May 26 14:56:02.206293 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 26 14:56:02.206293 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rec.execute(self) May 26 14:56:02.207211 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 26 14:56:02.207211 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 26 14:56:02.207211 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 26 14:56:02.207211 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application _emit_insert_statements( May 26 14:56:02.207211 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements May 26 14:56:02.207211 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application c = connection._execute_20( May 26 14:56:02.207211 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 26 14:56:02.207211 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 26 14:56:02.207211 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 26 14:56:02.207211 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 26 14:56:02.207211 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 26 14:56:02.207211 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ret = self._execute_context( May 26 14:56:02.207211 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 26 14:56:02.207211 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 26 14:56:02.207211 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 26 14:56:02.209276 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 26 14:56:02.209276 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 26 14:56:02.209276 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception May 26 14:56:02.209276 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 26 14:56:02.209276 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application self.dialect.do_execute( May 26 14:56:02.209276 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 26 14:56:02.209276 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 26 14:56:02.209276 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 26 14:56:02.209276 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application result = self._query(query) May 26 14:56:02.209276 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 26 14:56:02.209276 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application conn.query(q) May 26 14:56:02.209276 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 26 14:56:02.209276 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 26 14:56:02.209276 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 26 14:56:02.209276 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application result.read() May 26 14:56:02.209276 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 26 14:56:02.209276 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 26 14:56:02.211340 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 26 14:56:02.211340 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application packet.raise_for_error() May 26 14:56:02.211340 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 26 14:56:02.211340 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 26 14:56:02.211340 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 26 14:56:02.211340 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 26 14:56:02.211340 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 26 14:56:02.211340 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] May 26 14:56:02.211340 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [parameters: {'id': 'cc74a3e43aef4c6a8b00134f4bdff0e1', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] May 26 14:56:02.211340 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 26 14:56:02.211340 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 26 14:56:02.211340 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 14:56:02.211340 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 26 14:56:02.211340 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:56:02.211340 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project May 26 14:56:02.213255 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) May 26 14:56:02.213255 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 26 14:56:02.213255 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 26 14:56:02.213255 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. May 26 14:56:02.213255 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 26 14:56:02.213255 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 14:56:02.213255 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 26 14:56:02.213255 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:56:02.213255 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:56:02.213255 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:56:02.213255 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:56:02.213255 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:56:02.213255 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:56:02.213255 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:56:02.213255 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:56:02.213255 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:56:02.213255 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:56:02.219594 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:56:02.219594 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post May 26 14:56:02.219594 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( May 26 14:56:02.219594 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:56:02.219594 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:56:02.219594 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project May 26 14:56:02.219594 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.Conflict( May 26 14:56:02.219594 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id cc74a3e43aef4c6a8b00134f4bdff0e1. May 26 14:56:02.219594 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 26 14:56:02.219594 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 481/947] 10.222.0.46 () {60 vars in 1140 bytes} [Thu May 26 14:56:01 2022] POST /identity/v3/projects => generated 262 bytes in 210 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) May 26 14:56:02.220914 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e666b9-9fad-4f87-90e1-dee0605ca9b2 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:56:02.245868 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:56:02.322664 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e666b9-9fad-4f87-90e1-dee0605ca9b2 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:56:02.360583 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013f13c8-b952-4250-8f78-3ddf95bd079c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:56:02.361394 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013f13c8-b952-4250-8f78-3ddf95bd079c None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:02.361922 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013f13c8-b952-4250-8f78-3ddf95bd079c None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:02.362274 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013f13c8-b952-4250-8f78-3ddf95bd079c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:02.370240 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013f13c8-b952-4250-8f78-3ddf95bd079c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:56:02.371823 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013f13c8-b952-4250-8f78-3ddf95bd079c None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:56:02.390568 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-013f13c8-b952-4250-8f78-3ddf95bd079c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 14:56:02.390568 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:56:02.390568 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:56:02.390568 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:56:02.390568 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:56:02.390568 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:56:02.390568 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:56:02.390568 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:56:02.390568 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:56:02.390568 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:56:02.390568 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:56:02.390568 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:56:02.390568 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 14:56:02.390568 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 14:56:02.390568 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 14:56:02.393735 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 14:56:02.393735 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:56:02.393735 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:56:02.393735 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:56:02.393735 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:56:02.393735 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:56:02.393735 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:56:02.393735 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:56:02.393735 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 26 14:56:02.393735 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:56:02.393735 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 26 14:56:02.393735 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:56:02.393735 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:56:02.393735 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:56:02.393735 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 26 14:56:02.393735 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:56:02.393735 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 26 14:56:02.395824 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 14:56:02.395824 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 14:56:02.395824 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 14:56:02.395824 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 14:56:02.395824 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 14:56:02.395824 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 14:56:02.395824 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 14:56:02.395824 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 26 14:56:02.395824 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 467/948] 10.222.0.46 () {56 vars in 1111 bytes} [Thu May 26 14:56:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:56:02.401059 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aed4d26-b501-4315-ae75-0fa9f7580954 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:56:02.521587 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aed4d26-b501-4315-ae75-0fa9f7580954 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:56:02.557977 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d261ab20-c5ff-4782-b552-602cbf7e2e02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:56:02.558652 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d261ab20-c5ff-4782-b552-602cbf7e2e02 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:02.558939 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d261ab20-c5ff-4782-b552-602cbf7e2e02 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:02.559218 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d261ab20-c5ff-4782-b552-602cbf7e2e02 None admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:02.559988 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d261ab20-c5ff-4782-b552-602cbf7e2e02 None admin] RBAC: Adding query filter params (name=service) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:56:02.560299 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d261ab20-c5ff-4782-b552-602cbf7e2e02 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:56:02.561957 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d261ab20-c5ff-4782-b552-602cbf7e2e02 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:56:02.583577 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 482/949] 10.222.0.46 () {56 vars in 1112 bytes} [Thu May 26 14:56:02 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:56:03.365915 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ac6f8a-d1e5-491c-97ac-254929d3a536 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:03.366651 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ac6f8a-d1e5-491c-97ac-254929d3a536 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:03.366844 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ac6f8a-d1e5-491c-97ac-254929d3a536 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:03.369984 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 468/950] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 14:56:03 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 14:56:03.377802 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4f48dd-6526-440a-bf38-1a59293beb98 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:03.377802 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4f48dd-6526-440a-bf38-1a59293beb98 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:03.377802 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4f48dd-6526-440a-bf38-1a59293beb98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:03.395121 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-eb4f48dd-6526-440a-bf38-1a59293beb98 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:56:03.396953 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-eb4f48dd-6526-440a-bf38-1a59293beb98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:56:03.444863 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:56:03.477125 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 483/951] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 14:56:03 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 14:56:03.502689 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ac6f8a-d1e5-491c-97ac-254929d3a536 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:56:03.621929 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:56:03.636702 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ac6f8a-d1e5-491c-97ac-254929d3a536 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:56:03.653777 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9628fdd9-b74f-436f-99d3-c984f72cf00d None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:03.654123 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9628fdd9-b74f-436f-99d3-c984f72cf00d None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:03.654330 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9628fdd9-b74f-436f-99d3-c984f72cf00d None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:03.655959 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 484/952] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 14:56:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 14:56:03.661566 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6374501-dec4-4563-9cac-07767854d563 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:03.661874 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6374501-dec4-4563-9cac-07767854d563 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:03.662141 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6374501-dec4-4563-9cac-07767854d563 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:03.675515 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40bde69-2dd4-4efa-beed-ffb5bdd9d5f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:56:03.676672 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40bde69-2dd4-4efa-beed-ffb5bdd9d5f0 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:03.677141 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40bde69-2dd4-4efa-beed-ffb5bdd9d5f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:03.677635 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40bde69-2dd4-4efa-beed-ffb5bdd9d5f0 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:03.680285 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-b6374501-dec4-4563-9cac-07767854d563 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:56:03.682122 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-b6374501-dec4-4563-9cac-07767854d563 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:56:03.691816 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d40bde69-2dd4-4efa-beed-ffb5bdd9d5f0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 14:56:03.701344 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-d40bde69-2dd4-4efa-beed-ffb5bdd9d5f0 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 26 14:56:03.701344 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:56:03.701344 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:56:03.701344 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:56:03.701344 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:56:03.701344 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:56:03.701344 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:56:03.701344 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:56:03.701344 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:56:03.701344 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:56:03.701344 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:56:03.701344 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:56:03.701344 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 14:56:03.701344 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 14:56:03.701344 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 14:56:03.701344 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 14:56:03.702473 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 14:56:03.702473 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 14:56:03.702473 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 14:56:03.702473 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 14:56:03.702473 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:56:03.702473 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:56:03.702473 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:56:03.702473 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:56:03.702473 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:56:03.702473 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:56:03.702473 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:56:03.702473 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 26 14:56:03.702473 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:56:03.702473 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 26 14:56:03.702473 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:56:03.702473 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:56:03.703374 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:56:03.703374 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 26 14:56:03.703374 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:56:03.703374 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 26 14:56:03.703374 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 14:56:03.703374 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 14:56:03.703374 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 14:56:03.703374 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 14:56:03.703374 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 14:56:03.703374 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 14:56:03.703374 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 26 14:56:03.703374 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 26 14:56:03.705834 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 469/953] 10.222.0.46 () {56 vars in 1120 bytes} [Thu May 26 14:56:03 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:56:03.711393 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40bde69-2dd4-4efa-beed-ffb5bdd9d5f0 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:56:03.765719 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 485/954] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 14:56:03 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 26 14:56:03.785714 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6374501-dec4-4563-9cac-07767854d563 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:56:03.832277 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40bde69-2dd4-4efa-beed-ffb5bdd9d5f0 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:56:03.882375 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed725b9-ec5e-43de-abf5-1766cb5b2c51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:56:03.882989 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed725b9-ec5e-43de-abf5-1766cb5b2c51 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:03.883280 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed725b9-ec5e-43de-abf5-1766cb5b2c51 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:03.884116 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed725b9-ec5e-43de-abf5-1766cb5b2c51 None admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:03.884933 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed725b9-ec5e-43de-abf5-1766cb5b2c51 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:56:03.885256 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed725b9-ec5e-43de-abf5-1766cb5b2c51 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:56:03.886747 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed725b9-ec5e-43de-abf5-1766cb5b2c51 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:56:03.899107 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 470/955] 10.222.0.46 () {56 vars in 1115 bytes} [Thu May 26 14:56:03 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 14:56:03.910668 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed725b9-ec5e-43de-abf5-1766cb5b2c51 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:56:03.922844 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6374501-dec4-4563-9cac-07767854d563 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:56:03.930799 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:56:03.932157 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:56:03.972082 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb2a523-c0aa-4387-be07-1747c3fe1a5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:56:03.973152 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb2a523-c0aa-4387-be07-1747c3fe1a5a None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:03.973439 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb2a523-c0aa-4387-be07-1747c3fe1a5a None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:03.973744 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb2a523-c0aa-4387-be07-1747c3fe1a5a None admin] PATH_INFO: `/v3/services/compute` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:03.974940 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb2a523-c0aa-4387-be07-1747c3fe1a5a None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:56:03.976650 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb2a523-c0aa-4387-be07-1747c3fe1a5a None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:56:03.992244 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-2fb2a523-c0aa-4387-be07-1747c3fe1a5a None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 26 14:56:03.992244 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:56:03.992244 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 14:56:03.992244 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 14:56:03.992244 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 14:56:03.992244 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 14:56:03.992244 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 14:56:03.992244 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 14:56:03.992244 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 26 14:56:03.992244 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 26 14:56:03.992244 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 14:56:03.992244 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 26 14:56:03.992244 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:56:03.992244 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:56:03.992244 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:56:03.992244 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:56:03.992244 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:56:03.993982 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:56:03.993982 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:56:03.993982 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:56:03.993982 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:56:03.993982 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:56:03.993982 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:56:03.993982 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 14:56:03.993982 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 14:56:03.993982 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 14:56:03.993982 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 14:56:03.993982 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:56:03.993982 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:56:03.993982 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:56:03.993982 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:56:03.993982 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:56:03.993982 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:56:03.994865 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:56:03.994865 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 26 14:56:03.994865 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:56:03.994865 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 26 14:56:03.994865 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:56:03.994865 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:56:03.994865 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:56:03.994865 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 26 14:56:03.994865 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:56:03.994865 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 26 14:56:03.994865 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 14:56:03.994865 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 14:56:03.994865 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 26 14:56:03.994865 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 26 14:56:03.994865 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 486/956] 10.222.0.46 () {56 vars in 1111 bytes} [Thu May 26 14:56:03 2022] GET /identity/v3/services/compute => generated 88 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 14:56:03.999781 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb2a523-c0aa-4387-be07-1747c3fe1a5a None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:56:04.047499 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed725b9-ec5e-43de-abf5-1766cb5b2c51 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:56:04.096038 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8eff985-f83c-434f-86b1-7c4660f22915 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:56:04.097277 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8eff985-f83c-434f-86b1-7c4660f22915 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:04.097802 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8eff985-f83c-434f-86b1-7c4660f22915 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:04.098278 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8eff985-f83c-434f-86b1-7c4660f22915 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:04.111554 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8eff985-f83c-434f-86b1-7c4660f22915 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:56:04.113589 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8eff985-f83c-434f-86b1-7c4660f22915 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:56:04.135744 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-a8eff985-f83c-434f-86b1-7c4660f22915 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 14:56:04.135744 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:56:04.135744 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:56:04.135744 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:56:04.135744 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:56:04.135744 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:56:04.135744 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:56:04.135744 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:56:04.135744 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:56:04.135744 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:56:04.135744 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:56:04.135744 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:56:04.135744 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 14:56:04.135744 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 14:56:04.135744 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 14:56:04.136675 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 14:56:04.136675 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:56:04.136675 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:56:04.136675 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:56:04.136675 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:56:04.136675 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:56:04.136675 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:56:04.136675 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:56:04.136675 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 26 14:56:04.136675 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:56:04.136675 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 26 14:56:04.136675 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:56:04.136675 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:56:04.136675 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:56:04.136675 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 26 14:56:04.136675 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:56:04.136675 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 26 14:56:04.138002 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 14:56:04.138002 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 14:56:04.138002 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 14:56:04.138002 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 26 14:56:04.140050 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 471/957] 10.222.0.46 () {56 vars in 1108 bytes} [Thu May 26 14:56:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 14:56:04.146560 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8eff985-f83c-434f-86b1-7c4660f22915 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:56:04.154078 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb2a523-c0aa-4387-be07-1747c3fe1a5a None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:56:04.190963 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4bf370-2ca4-46ca-8bc3-426105cc4b9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:56:04.191635 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4bf370-2ca4-46ca-8bc3-426105cc4b9a None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:04.191915 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4bf370-2ca4-46ca-8bc3-426105cc4b9a None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:04.192190 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4bf370-2ca4-46ca-8bc3-426105cc4b9a None admin] PATH_INFO: `/v3/services` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:04.192897 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4bf370-2ca4-46ca-8bc3-426105cc4b9a None admin] RBAC: Adding query filter params (name=compute) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:56:04.193201 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4bf370-2ca4-46ca-8bc3-426105cc4b9a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:56:04.194585 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4bf370-2ca4-46ca-8bc3-426105cc4b9a None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:56:04.204982 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 487/958] 10.222.0.46 () {56 vars in 1112 bytes} [Thu May 26 14:56:03 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 14:56:04.206072 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4bf370-2ca4-46ca-8bc3-426105cc4b9a None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:56:04.246415 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:56:04.247171 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:56:04.298824 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8eff985-f83c-434f-86b1-7c4660f22915 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:56:04.321117 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4bf370-2ca4-46ca-8bc3-426105cc4b9a None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:56:04.341964 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cfd92b-a7d9-4b39-8705-69d056f08ddb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:56:04.342642 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cfd92b-a7d9-4b39-8705-69d056f08ddb None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:04.342923 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cfd92b-a7d9-4b39-8705-69d056f08ddb None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:04.343201 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cfd92b-a7d9-4b39-8705-69d056f08ddb None admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:04.343924 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51cfd92b-a7d9-4b39-8705-69d056f08ddb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:56:04.344237 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51cfd92b-a7d9-4b39-8705-69d056f08ddb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:56:04.345637 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51cfd92b-a7d9-4b39-8705-69d056f08ddb None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:56:04.360388 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f721e311-adf9-491b-855f-294d618a38c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:56:04.361063 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f721e311-adf9-491b-855f-294d618a38c2 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:04.361349 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f721e311-adf9-491b-855f-294d618a38c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:04.361653 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f721e311-adf9-491b-855f-294d618a38c2 None admin] PATH_INFO: `/v3/services` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:04.366285 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 472/959] 10.222.0.46 () {56 vars in 1109 bytes} [Thu May 26 14:56:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:56:04.366737 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f721e311-adf9-491b-855f-294d618a38c2 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:56:04.366737 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f721e311-adf9-491b-855f-294d618a38c2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:56:04.366873 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715fbce7-1c2c-4f79-8f64-522f069c7157 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:04.366873 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715fbce7-1c2c-4f79-8f64-522f069c7157 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:04.366873 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715fbce7-1c2c-4f79-8f64-522f069c7157 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:04.367092 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f721e311-adf9-491b-855f-294d618a38c2 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:56:04.367164 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 473/960] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 14:56:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 14:56:04.368675 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715fbce7-1c2c-4f79-8f64-522f069c7157 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:56:04.376932 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 488/961] 10.222.0.46 () {56 vars in 1112 bytes} [Thu May 26 14:56:04 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 14:56:04.379009 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe2ba41-cf55-441f-b043-a709104dbfcc None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:04.379808 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe2ba41-cf55-441f-b043-a709104dbfcc None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:04.380294 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe2ba41-cf55-441f-b043-a709104dbfcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:04.407688 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-2fe2ba41-cf55-441f-b043-a709104dbfcc None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:56:04.409993 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-2fe2ba41-cf55-441f-b043-a709104dbfcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:56:04.514636 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715fbce7-1c2c-4f79-8f64-522f069c7157 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:56:04.514999 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 489/962] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 14:56:04 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 14:56:04.537508 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe2ba41-cf55-441f-b043-a709104dbfcc None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:56:04.557890 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:56:04.568503 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06818d4-ee7c-4039-b2b9-4aaf2d84aa43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:56:04.569467 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06818d4-ee7c-4039-b2b9-4aaf2d84aa43 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:04.569791 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06818d4-ee7c-4039-b2b9-4aaf2d84aa43 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:04.570068 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06818d4-ee7c-4039-b2b9-4aaf2d84aa43 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:04.578375 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:56:04.578571 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06818d4-ee7c-4039-b2b9-4aaf2d84aa43 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:56:04.579938 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06818d4-ee7c-4039-b2b9-4aaf2d84aa43 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:56:04.597320 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-e06818d4-ee7c-4039-b2b9-4aaf2d84aa43 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 26 14:56:04.597320 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:56:04.597320 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:56:04.597320 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:56:04.597320 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:56:04.597320 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:56:04.597320 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:56:04.597320 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:56:04.597320 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:56:04.597320 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:56:04.597320 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:56:04.597320 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:56:04.597320 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 14:56:04.597320 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 14:56:04.597320 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 14:56:04.597320 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 14:56:04.598563 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:56:04.598563 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:56:04.598563 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 14:56:04.598563 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 14:56:04.598563 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 14:56:04.598563 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 14:56:04.598563 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:56:04.598563 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:56:04.598563 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:56:04.598563 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:56:04.598563 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:56:04.598563 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 26 14:56:04.598563 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:56:04.598563 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 26 14:56:04.598563 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:56:04.598563 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:56:04.598563 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:56:04.598563 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 26 14:56:04.599611 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:56:04.599611 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 26 14:56:04.599611 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 14:56:04.599611 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 14:56:04.599611 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 14:56:04.599611 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 14:56:04.599611 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 14:56:04.599611 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 14:56:04.599611 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 26 14:56:04.599611 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 26 14:56:04.599611 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 474/963] 10.222.0.46 () {56 vars in 1117 bytes} [Thu May 26 14:56:04 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:56:04.605404 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06818d4-ee7c-4039-b2b9-4aaf2d84aa43 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:56:04.673806 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe2ba41-cf55-441f-b043-a709104dbfcc None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:56:04.711142 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa537ae3-a9d8-4f83-a5c5-8d466b19ea06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:56:04.712932 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa537ae3-a9d8-4f83-a5c5-8d466b19ea06 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:04.713552 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa537ae3-a9d8-4f83-a5c5-8d466b19ea06 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:04.714035 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa537ae3-a9d8-4f83-a5c5-8d466b19ea06 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:04.714912 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa537ae3-a9d8-4f83-a5c5-8d466b19ea06 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:56:04.716600 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa537ae3-a9d8-4f83-a5c5-8d466b19ea06 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:56:04.730769 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-aa537ae3-a9d8-4f83-a5c5-8d466b19ea06 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 26 14:56:04.730769 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:56:04.730769 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 14:56:04.730769 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 14:56:04.730769 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 14:56:04.730769 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 14:56:04.730769 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 14:56:04.730769 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 14:56:04.730769 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 26 14:56:04.730769 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 26 14:56:04.730769 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 14:56:04.730769 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 26 14:56:04.730769 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:56:04.730769 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:56:04.730769 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:56:04.730769 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:56:04.730769 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:56:04.732356 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:56:04.732356 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:56:04.732356 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:56:04.732356 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:56:04.732356 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:56:04.732356 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:56:04.732356 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 14:56:04.732356 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 14:56:04.732356 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 14:56:04.732356 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 14:56:04.732356 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:56:04.732356 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:56:04.732356 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:56:04.732356 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:56:04.732356 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:56:04.732356 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:56:04.733184 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:56:04.733184 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 26 14:56:04.733184 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:56:04.733184 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 26 14:56:04.733184 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:56:04.733184 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:56:04.733184 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:56:04.733184 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 26 14:56:04.733184 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:56:04.733184 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 26 14:56:04.733184 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 14:56:04.733184 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 14:56:04.733184 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 26 14:56:04.733184 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 26 14:56:04.736441 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 490/964] 10.222.0.46 () {56 vars in 1126 bytes} [Thu May 26 14:56:04 2022] GET /identity/v3/services/object-store => generated 93 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:56:04.742412 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa537ae3-a9d8-4f83-a5c5-8d466b19ea06 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:56:04.759135 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06818d4-ee7c-4039-b2b9-4aaf2d84aa43 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:56:04.815357 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c53683-af27-42cd-a89c-227814bb8e7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:56:04.816027 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c53683-af27-42cd-a89c-227814bb8e7b None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:04.816307 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c53683-af27-42cd-a89c-227814bb8e7b None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:04.816579 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c53683-af27-42cd-a89c-227814bb8e7b None admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:04.817416 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c53683-af27-42cd-a89c-227814bb8e7b None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:56:04.817752 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c53683-af27-42cd-a89c-227814bb8e7b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:56:04.819397 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c53683-af27-42cd-a89c-227814bb8e7b None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:56:04.840383 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 475/965] 10.222.0.46 () {56 vars in 1149 bytes} [Thu May 26 14:56:04 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 14:56:04.845876 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c53683-af27-42cd-a89c-227814bb8e7b None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:56:04.863675 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:56:04.878407 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa537ae3-a9d8-4f83-a5c5-8d466b19ea06 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:56:04.900192 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:56:04.930103 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c6b247-ea54-4c33-8c21-863312e21037 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:56:04.931154 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c6b247-ea54-4c33-8c21-863312e21037 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:04.931649 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c6b247-ea54-4c33-8c21-863312e21037 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:04.932134 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c6b247-ea54-4c33-8c21-863312e21037 None admin] PATH_INFO: `/v3/services` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:04.933628 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c6b247-ea54-4c33-8c21-863312e21037 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:56:04.934158 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c6b247-ea54-4c33-8c21-863312e21037 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:56:04.937791 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c6b247-ea54-4c33-8c21-863312e21037 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:56:04.950211 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 491/966] 10.222.0.46 () {56 vars in 1122 bytes} [Thu May 26 14:56:04 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:56:04.951451 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c6b247-ea54-4c33-8c21-863312e21037 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:56:04.962898 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c53683-af27-42cd-a89c-227814bb8e7b None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:56:04.999590 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3deb206-2569-4d24-887a-8e886c0ec835 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:56:05.000266 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3deb206-2569-4d24-887a-8e886c0ec835 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:05.000649 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3deb206-2569-4d24-887a-8e886c0ec835 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:05.000929 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3deb206-2569-4d24-887a-8e886c0ec835 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:05.062185 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c6b247-ea54-4c33-8c21-863312e21037 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:56:05.076638 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3deb206-2569-4d24-887a-8e886c0ec835 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:56:05.079285 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3deb206-2569-4d24-887a-8e886c0ec835 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:56:05.105887 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21873636-4e4d-45b5-9788-190492f5afcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:56:05.106562 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21873636-4e4d-45b5-9788-190492f5afcc None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:05.106849 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21873636-4e4d-45b5-9788-190492f5afcc None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:05.107128 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21873636-4e4d-45b5-9788-190492f5afcc None admin] PATH_INFO: `/v3/services` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:05.107848 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21873636-4e4d-45b5-9788-190492f5afcc None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:56:05.108158 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21873636-4e4d-45b5-9788-190492f5afcc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:56:05.109543 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21873636-4e4d-45b5-9788-190492f5afcc None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:56:05.121686 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 492/967] 10.222.0.46 () {56 vars in 1122 bytes} [Thu May 26 14:56:04 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:56:05.121686 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ee0855-4856-4a68-ba86-84d9b600b4cb None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:05.121686 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ee0855-4856-4a68-ba86-84d9b600b4cb None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:05.121686 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ee0855-4856-4a68-ba86-84d9b600b4cb None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:05.121686 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 493/968] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 14:56:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 14:56:05.121686 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ee0855-4856-4a68-ba86-84d9b600b4cb None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:56:05.144306 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:56:05.145913 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:56:05.212680 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 476/969] 10.222.0.46 () {58 vars in 1282 bytes} [Thu May 26 14:56:04 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 26 14:56:05.214427 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83a04ac-80ba-4628-91b5-dd93042e5127 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:05.214722 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83a04ac-80ba-4628-91b5-dd93042e5127 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:05.215003 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83a04ac-80ba-4628-91b5-dd93042e5127 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:05.233109 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-c83a04ac-80ba-4628-91b5-dd93042e5127 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:56:05.234456 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-c83a04ac-80ba-4628-91b5-dd93042e5127 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:56:05.237615 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ee0855-4856-4a68-ba86-84d9b600b4cb None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:56:05.275585 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208cdbdf-a346-4735-a36c-bce9daf0cd28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:56:05.276257 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208cdbdf-a346-4735-a36c-bce9daf0cd28 None admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:05.276546 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208cdbdf-a346-4735-a36c-bce9daf0cd28 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:05.276829 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208cdbdf-a346-4735-a36c-bce9daf0cd28 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:05.277611 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208cdbdf-a346-4735-a36c-bce9daf0cd28 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:56:05.278992 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-208cdbdf-a346-4735-a36c-bce9daf0cd28 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:56:05.308525 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 494/970] 10.222.0.46 () {60 vars in 1144 bytes} [Thu May 26 14:56:05 2022] POST /identity/v3/endpoints => generated 349 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 14:56:05.309347 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208cdbdf-a346-4735-a36c-bce9daf0cd28 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:56:05.341487 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 477/971] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 14:56:05 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 14:56:05.360189 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83a04ac-80ba-4628-91b5-dd93042e5127 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:56:05.425316 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:56:05.458984 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208cdbdf-a346-4735-a36c-bce9daf0cd28 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:56:05.471932 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:56:05.481379 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83a04ac-80ba-4628-91b5-dd93042e5127 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:56:05.506134 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884f79d3-d590-4711-95c6-0bc281235dd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:56:05.507381 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884f79d3-d590-4711-95c6-0bc281235dd6 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:05.507666 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884f79d3-d590-4711-95c6-0bc281235dd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:05.508097 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884f79d3-d590-4711-95c6-0bc281235dd6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:05.521075 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884f79d3-d590-4711-95c6-0bc281235dd6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:56:05.522329 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-884f79d3-d590-4711-95c6-0bc281235dd6 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:56:05.534268 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c64a41a-a071-49c3-bf4a-897017b4992b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:56:05.534662 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c64a41a-a071-49c3-bf4a-897017b4992b None admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:05.534949 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c64a41a-a071-49c3-bf4a-897017b4992b None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:05.535228 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c64a41a-a071-49c3-bf4a-897017b4992b None admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:05.535944 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c64a41a-a071-49c3-bf4a-897017b4992b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:56:05.537382 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c64a41a-a071-49c3-bf4a-897017b4992b None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:56:05.539335 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: WARNING keystone.server.flask.common [None req-0c64a41a-a071-49c3-bf4a-897017b4992b None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. May 26 14:56:05.542881 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-884f79d3-d590-4711-95c6-0bc281235dd6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 14:56:05.542881 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:56:05.542881 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:56:05.542881 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:56:05.542881 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:56:05.542881 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:56:05.542881 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:56:05.542881 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:56:05.542881 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:56:05.542881 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:56:05.542881 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:56:05.542881 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:56:05.542881 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 14:56:05.542881 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 14:56:05.542881 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 14:56:05.543773 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 14:56:05.543773 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:56:05.543773 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:56:05.543773 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:56:05.543773 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:56:05.543773 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:56:05.543773 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:56:05.543773 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:56:05.543773 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 26 14:56:05.543773 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:56:05.543773 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 26 14:56:05.543773 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:56:05.543773 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:56:05.543773 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:56:05.543773 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 26 14:56:05.543773 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:56:05.543773 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 26 14:56:05.544702 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 14:56:05.544702 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 14:56:05.544702 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 14:56:05.544702 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 26 14:56:05.544702 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 495/972] 10.222.0.46 () {56 vars in 1108 bytes} [Thu May 26 14:56:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:56:05.552344 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884f79d3-d590-4711-95c6-0bc281235dd6 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:56:05.556172 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.sql.core [None req-0c64a41a-a071-49c3-bf4a-897017b4992b None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 26 14:56:05.556172 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] May 26 14:56:05.556172 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [parameters: {'user_id': 'ae2e06c6f46046dfb6cc7dde11e603f7', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] May 26 14:56:05.556172 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80167) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 26 14:56:05.573365 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-0c64a41a-a071-49c3-bf4a-897017b4992b None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. May 26 14:56:05.573365 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:56:05.573365 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 26 14:56:05.573365 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application self.dialect.do_execute( May 26 14:56:05.573365 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 26 14:56:05.573365 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 26 14:56:05.573365 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 26 14:56:05.573365 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application result = self._query(query) May 26 14:56:05.573365 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 26 14:56:05.573365 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application conn.query(q) May 26 14:56:05.573365 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 26 14:56:05.573365 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 26 14:56:05.573365 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 26 14:56:05.573365 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application result.read() May 26 14:56:05.573365 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 26 14:56:05.574392 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 26 14:56:05.574392 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 26 14:56:05.574392 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application packet.raise_for_error() May 26 14:56:05.574392 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 26 14:56:05.574392 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 26 14:56:05.574392 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 26 14:56:05.574392 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 26 14:56:05.574392 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 26 14:56:05.574392 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 26 14:56:05.574392 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 26 14:56:05.574392 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 26 14:56:05.574392 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:56:05.574392 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 26 14:56:05.574392 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 26 14:56:05.574392 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user May 26 14:56:05.574392 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) May 26 14:56:05.574392 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 26 14:56:05.574392 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application next(self.gen) May 26 14:56:05.575707 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 26 14:56:05.575707 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application yield resource May 26 14:56:05.575707 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 26 14:56:05.575707 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application next(self.gen) May 26 14:56:05.575707 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 26 14:56:05.575707 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application self.session.rollback() May 26 14:56:05.575707 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 26 14:56:05.575707 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application self.force_reraise() May 26 14:56:05.575707 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 26 14:56:05.575707 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise self.value May 26 14:56:05.575707 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 26 14:56:05.575707 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 26 14:56:05.575707 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 26 14:56:05.575707 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application session.commit() May 26 14:56:05.575707 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 26 14:56:05.575707 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 26 14:56:05.575707 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 26 14:56:05.576810 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application self._prepare_impl() May 26 14:56:05.576810 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 26 14:56:05.576810 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application self.session.flush() May 26 14:56:05.576810 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 26 14:56:05.576810 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application self._flush(objects) May 26 14:56:05.576810 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 26 14:56:05.576810 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 26 14:56:05.576810 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 26 14:56:05.576810 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application compat.raise_( May 26 14:56:05.576810 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 26 14:56:05.576810 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception May 26 14:56:05.576810 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 26 14:56:05.576810 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application flush_context.execute() May 26 14:56:05.576810 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 26 14:56:05.576810 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rec.execute(self) May 26 14:56:05.576810 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 26 14:56:05.576810 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 26 14:56:05.577803 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 26 14:56:05.577803 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application _emit_insert_statements( May 26 14:56:05.577803 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements May 26 14:56:05.577803 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application result = connection._execute_20( May 26 14:56:05.577803 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 26 14:56:05.577803 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 26 14:56:05.577803 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 26 14:56:05.577803 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 26 14:56:05.577803 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 26 14:56:05.577803 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ret = self._execute_context( May 26 14:56:05.577803 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 26 14:56:05.577803 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 26 14:56:05.577803 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 26 14:56:05.577803 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 26 14:56:05.577803 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 26 14:56:05.577803 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception May 26 14:56:05.578660 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 26 14:56:05.578660 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application self.dialect.do_execute( May 26 14:56:05.578660 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 26 14:56:05.578660 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 26 14:56:05.578660 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 26 14:56:05.578660 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application result = self._query(query) May 26 14:56:05.578660 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 26 14:56:05.578660 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application conn.query(q) May 26 14:56:05.578660 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 26 14:56:05.578660 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 26 14:56:05.578660 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 26 14:56:05.578660 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application result.read() May 26 14:56:05.578660 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 26 14:56:05.578660 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 26 14:56:05.578660 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 26 14:56:05.578660 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application packet.raise_for_error() May 26 14:56:05.580180 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 26 14:56:05.580180 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 26 14:56:05.580180 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 26 14:56:05.580180 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 26 14:56:05.580180 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 26 14:56:05.580180 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] May 26 14:56:05.580180 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [parameters: {'user_id': 'ae2e06c6f46046dfb6cc7dde11e603f7', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] May 26 14:56:05.580180 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 26 14:56:05.580180 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 26 14:56:05.580180 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 14:56:05.580180 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 26 14:56:05.580180 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:56:05.580180 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:56:05.580180 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:56:05.580180 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:56:05.581169 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:56:05.581169 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:56:05.581169 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:56:05.581169 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:56:05.581169 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:56:05.581169 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:56:05.581169 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:56:05.581169 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post May 26 14:56:05.581169 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( May 26 14:56:05.581169 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:56:05.581169 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:56:05.581169 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 14:56:05.581169 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 14:56:05.581169 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 14:56:05.581169 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 14:56:05.581169 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user May 26 14:56:05.581169 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) May 26 14:56:05.583355 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects May 26 14:56:05.583355 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) May 26 14:56:05.583355 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 26 14:56:05.583355 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 26 14:56:05.583355 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. May 26 14:56:05.583355 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 26 14:56:05.583355 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 478/973] 10.222.0.46 () {60 vars in 1131 bytes} [Thu May 26 14:56:05 2022] POST /identity/v3/users => generated 160 bytes in 215 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) May 26 14:56:05.584636 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c64a41a-a071-49c3-bf4a-897017b4992b None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:56:05.685193 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884f79d3-d590-4711-95c6-0bc281235dd6 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:56:05.715354 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:56:05.719883 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c64a41a-a071-49c3-bf4a-897017b4992b None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:56:05.726208 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da637fbe-5d30-4080-a42f-c33a7fd805d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:56:05.727110 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da637fbe-5d30-4080-a42f-c33a7fd805d2 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:05.727561 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da637fbe-5d30-4080-a42f-c33a7fd805d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:05.727975 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da637fbe-5d30-4080-a42f-c33a7fd805d2 None admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:05.729055 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da637fbe-5d30-4080-a42f-c33a7fd805d2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:56:05.729545 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da637fbe-5d30-4080-a42f-c33a7fd805d2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:56:05.732478 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da637fbe-5d30-4080-a42f-c33a7fd805d2 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:56:05.748784 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 496/974] 10.222.0.46 () {56 vars in 1109 bytes} [Thu May 26 14:56:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:56:05.755459 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:56:05.761474 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da637fbe-5d30-4080-a42f-c33a7fd805d2 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:56:05.761973 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06458e5-8ffb-44c7-b0df-d5acef66105c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:56:05.763059 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06458e5-8ffb-44c7-b0df-d5acef66105c None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:05.763430 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06458e5-8ffb-44c7-b0df-d5acef66105c None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:05.764656 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06458e5-8ffb-44c7-b0df-d5acef66105c None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:05.775266 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06458e5-8ffb-44c7-b0df-d5acef66105c None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:56:05.777677 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06458e5-8ffb-44c7-b0df-d5acef66105c None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:56:05.796113 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-e06458e5-8ffb-44c7-b0df-d5acef66105c None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 26 14:56:05.796113 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:56:05.796113 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:56:05.796113 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:56:05.796113 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:56:05.796113 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:56:05.796113 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:56:05.796113 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:56:05.796113 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:56:05.796113 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:56:05.796113 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:56:05.796113 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:56:05.796113 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 14:56:05.796113 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 14:56:05.796113 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 14:56:05.796113 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 14:56:05.797301 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:56:05.797301 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:56:05.797301 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 14:56:05.797301 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 14:56:05.797301 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 14:56:05.797301 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 14:56:05.797301 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:56:05.797301 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:56:05.797301 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:56:05.797301 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:56:05.797301 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:56:05.797301 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 26 14:56:05.797301 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:56:05.797301 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 26 14:56:05.797301 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:56:05.797301 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:56:05.797301 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:56:05.797301 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 26 14:56:05.798411 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:56:05.798411 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 26 14:56:05.798411 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 14:56:05.798411 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 14:56:05.798411 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 14:56:05.798411 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 14:56:05.798411 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 14:56:05.798411 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 14:56:05.798411 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 26 14:56:05.798411 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 26 14:56:05.798411 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 479/975] 10.222.0.46 () {56 vars in 1099 bytes} [Thu May 26 14:56:05 2022] GET /identity/v3/users/cinder => generated 84 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 14:56:05.803820 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06458e5-8ffb-44c7-b0df-d5acef66105c None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:56:05.886132 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da637fbe-5d30-4080-a42f-c33a7fd805d2 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:56:05.919408 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06458e5-8ffb-44c7-b0df-d5acef66105c None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:56:05.928824 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd726f3-c5fa-4ea3-8d0d-1c72c2587d28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:56:05.929912 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd726f3-c5fa-4ea3-8d0d-1c72c2587d28 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:05.931485 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd726f3-c5fa-4ea3-8d0d-1c72c2587d28 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:05.931870 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd726f3-c5fa-4ea3-8d0d-1c72c2587d28 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:05.942196 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd726f3-c5fa-4ea3-8d0d-1c72c2587d28 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:56:05.945157 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd726f3-c5fa-4ea3-8d0d-1c72c2587d28 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:56:05.963367 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab6a5a3-4f33-467b-84d7-393cea13d9f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:56:05.964192 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab6a5a3-4f33-467b-84d7-393cea13d9f0 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:05.964576 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab6a5a3-4f33-467b-84d7-393cea13d9f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:05.964956 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab6a5a3-4f33-467b-84d7-393cea13d9f0 None admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:05.965964 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab6a5a3-4f33-467b-84d7-393cea13d9f0 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:56:05.966093 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-efd726f3-c5fa-4ea3-8d0d-1c72c2587d28 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 14:56:05.966093 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:56:05.966093 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:56:05.966093 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:56:05.966093 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:56:05.966093 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:56:05.966093 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:56:05.966093 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:56:05.966093 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:56:05.966093 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:56:05.966093 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:56:05.966093 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:56:05.966093 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 14:56:05.966093 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 14:56:05.966093 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 14:56:05.967109 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 14:56:05.967109 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:56:05.967109 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:56:05.967109 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:56:05.967109 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:56:05.967109 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:56:05.967109 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:56:05.967109 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:56:05.967109 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 26 14:56:05.967109 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:56:05.967109 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 26 14:56:05.967109 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:56:05.967109 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:56:05.967109 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:56:05.967109 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 26 14:56:05.967109 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:56:05.967109 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 26 14:56:05.968221 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 14:56:05.968221 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 14:56:05.968221 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 14:56:05.968221 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 14:56:05.968221 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 14:56:05.968221 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 14:56:05.968221 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 14:56:05.968221 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 26 14:56:05.968748 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab6a5a3-4f33-467b-84d7-393cea13d9f0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:56:05.970469 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab6a5a3-4f33-467b-84d7-393cea13d9f0 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:56:05.970628 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 497/976] 10.222.0.46 () {56 vars in 1111 bytes} [Thu May 26 14:56:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:56:05.978535 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd726f3-c5fa-4ea3-8d0d-1c72c2587d28 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:56:06.012735 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 480/977] 10.222.0.46 () {56 vars in 1101 bytes} [Thu May 26 14:56:05 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:56:06.057436 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:56:06.097218 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd726f3-c5fa-4ea3-8d0d-1c72c2587d28 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:56:06.138712 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb557640-679c-4a4b-ad47-3c1a4939eb24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:56:06.139554 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb557640-679c-4a4b-ad47-3c1a4939eb24 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:06.139943 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb557640-679c-4a4b-ad47-3c1a4939eb24 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:06.140341 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb557640-679c-4a4b-ad47-3c1a4939eb24 None admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:06.141251 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb557640-679c-4a4b-ad47-3c1a4939eb24 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:56:06.141710 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb557640-679c-4a4b-ad47-3c1a4939eb24 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:56:06.143332 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb557640-679c-4a4b-ad47-3c1a4939eb24 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:56:06.158922 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 498/978] 10.222.0.46 () {56 vars in 1148 bytes} [Thu May 26 14:56:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:56:06.166659 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab6a5a3-4f33-467b-84d7-393cea13d9f0 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:56:06.223248 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:56:06.278900 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab6a5a3-4f33-467b-84d7-393cea13d9f0 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:56:06.316182 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76535f97-30e2-4e4a-8e95-4eb5586352e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:56:06.316851 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76535f97-30e2-4e4a-8e95-4eb5586352e7 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:06.317134 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76535f97-30e2-4e4a-8e95-4eb5586352e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:06.317413 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76535f97-30e2-4e4a-8e95-4eb5586352e7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:06.318286 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76535f97-30e2-4e4a-8e95-4eb5586352e7 None admin] RBAC: Adding query filter params (role.id=3593816f89a648809d5d7c9160fb87dc, scope.project.id=a17fc8227d4c4da6b2847b350e912ee3, user.id=8028c520d93449a6a087c281280ee13d) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:56:06.318600 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76535f97-30e2-4e4a-8e95-4eb5586352e7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:56:06.320032 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76535f97-30e2-4e4a-8e95-4eb5586352e7 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:56:06.331942 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 481/979] 10.222.0.46 () {56 vars in 1374 bytes} [Thu May 26 14:56:06 2022] GET /identity/v3/role_assignments?user.id=8028c520d93449a6a087c281280ee13d&scope.project.id=a17fc8227d4c4da6b2847b350e912ee3&role.id=3593816f89a648809d5d7c9160fb87dc => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:56:06.690961 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cfa5fe-3d39-4f0a-8a8d-9a06693fb14d None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:06.691497 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cfa5fe-3d39-4f0a-8a8d-9a06693fb14d None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:06.691497 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cfa5fe-3d39-4f0a-8a8d-9a06693fb14d None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:06.693023 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 499/980] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 14:56:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 14:56:06.698137 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d7ba06-f08d-42d7-84c7-d0eaba04214c None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:06.698557 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d7ba06-f08d-42d7-84c7-d0eaba04214c None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:06.698723 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d7ba06-f08d-42d7-84c7-d0eaba04214c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:06.715809 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-16d7ba06-f08d-42d7-84c7-d0eaba04214c None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:56:06.717297 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-16d7ba06-f08d-42d7-84c7-d0eaba04214c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:56:06.784818 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 482/981] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 14:56:06 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 14:56:06.806883 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cfa5fe-3d39-4f0a-8a8d-9a06693fb14d None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:56:06.912745 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cfa5fe-3d39-4f0a-8a8d-9a06693fb14d None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:56:06.922542 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:56:06.948874 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8844c66e-9056-47c6-9f7f-9c479e79ab84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:56:06.949563 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8844c66e-9056-47c6-9f7f-9c479e79ab84 None admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:06.949847 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8844c66e-9056-47c6-9f7f-9c479e79ab84 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:06.950122 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8844c66e-9056-47c6-9f7f-9c479e79ab84 None admin] PATH_INFO: `/v3/services` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:06.950847 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8844c66e-9056-47c6-9f7f-9c479e79ab84 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:56:06.952201 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8844c66e-9056-47c6-9f7f-9c479e79ab84 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:56:06.973493 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 500/982] 10.222.0.46 () {60 vars in 1141 bytes} [Thu May 26 14:56:06 2022] POST /identity/v3/services => generated 242 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 14:56:07.640217 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e41886-def3-4aed-adda-5cf977e02b5d None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:07.640529 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e41886-def3-4aed-adda-5cf977e02b5d None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:07.640782 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e41886-def3-4aed-adda-5cf977e02b5d None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:07.642321 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 483/983] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 14:56:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 14:56:07.647753 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30d4828-0586-4a54-871a-75e40c2225ed None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:07.648050 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30d4828-0586-4a54-871a-75e40c2225ed None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:07.648329 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30d4828-0586-4a54-871a-75e40c2225ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:07.666013 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-b30d4828-0586-4a54-871a-75e40c2225ed None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:56:07.667189 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-b30d4828-0586-4a54-871a-75e40c2225ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:56:07.745983 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 501/984] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 14:56:07 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 14:56:07.768212 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e41886-def3-4aed-adda-5cf977e02b5d None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:56:07.818360 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:56:07.877061 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e41886-def3-4aed-adda-5cf977e02b5d None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:56:07.911028 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf81de6-9b87-4a8f-8d8d-db0a55b53d9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:56:07.911834 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf81de6-9b87-4a8f-8d8d-db0a55b53d9e None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:07.912117 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf81de6-9b87-4a8f-8d8d-db0a55b53d9e None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:07.912394 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf81de6-9b87-4a8f-8d8d-db0a55b53d9e None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:07.913043 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf81de6-9b87-4a8f-8d8d-db0a55b53d9e None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:56:07.914439 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf81de6-9b87-4a8f-8d8d-db0a55b53d9e None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:56:07.927117 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-dcf81de6-9b87-4a8f-8d8d-db0a55b53d9e None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 26 14:56:07.927117 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:56:07.927117 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 14:56:07.927117 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 14:56:07.927117 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 14:56:07.927117 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 14:56:07.927117 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 14:56:07.927117 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 14:56:07.927117 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 26 14:56:07.927117 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 26 14:56:07.927117 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 14:56:07.927117 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 26 14:56:07.927117 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:56:07.927117 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:56:07.927117 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:56:07.927117 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:56:07.927117 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:56:07.928296 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:56:07.928296 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:56:07.928296 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:56:07.928296 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:56:07.928296 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:56:07.928296 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:56:07.928296 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 14:56:07.928296 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 14:56:07.928296 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 14:56:07.928296 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 14:56:07.928296 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:56:07.928296 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:56:07.928296 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:56:07.928296 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:56:07.928296 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:56:07.928296 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:56:07.934137 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:56:07.934137 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 26 14:56:07.934137 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:56:07.934137 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 26 14:56:07.934137 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:56:07.934137 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:56:07.934137 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:56:07.934137 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 26 14:56:07.934137 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:56:07.934137 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 26 14:56:07.934137 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 14:56:07.934137 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 14:56:07.934137 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 26 14:56:07.934137 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 26 14:56:07.934137 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 484/985] 10.222.0.46 () {56 vars in 1126 bytes} [Thu May 26 14:56:07 2022] GET /identity/v3/services/object-store => generated 93 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:56:07.936499 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30d4828-0586-4a54-871a-75e40c2225ed None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:56:08.040706 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:56:08.049072 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30d4828-0586-4a54-871a-75e40c2225ed None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:56:08.084704 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63970ee4-6c4f-456c-a5ac-bcb3e29c67c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:56:08.085375 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63970ee4-6c4f-456c-a5ac-bcb3e29c67c0 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:08.085728 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63970ee4-6c4f-456c-a5ac-bcb3e29c67c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:08.085966 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63970ee4-6c4f-456c-a5ac-bcb3e29c67c0 None admin] PATH_INFO: `/v3/services` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:08.087320 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63970ee4-6c4f-456c-a5ac-bcb3e29c67c0 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:56:08.087625 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63970ee4-6c4f-456c-a5ac-bcb3e29c67c0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:56:08.089000 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63970ee4-6c4f-456c-a5ac-bcb3e29c67c0 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:56:08.100128 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 502/986] 10.222.0.46 () {56 vars in 1122 bytes} [Thu May 26 14:56:07 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 14:56:08.102225 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf81de6-9b87-4a8f-8d8d-db0a55b53d9e None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:56:08.228885 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf81de6-9b87-4a8f-8d8d-db0a55b53d9e None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:56:08.238009 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:56:08.264238 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc78d38-b139-43d0-92dd-6e4a550f2d56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:56:08.264909 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc78d38-b139-43d0-92dd-6e4a550f2d56 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:08.265197 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc78d38-b139-43d0-92dd-6e4a550f2d56 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:08.265505 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc78d38-b139-43d0-92dd-6e4a550f2d56 None admin] PATH_INFO: `/v3/services` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:08.266223 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc78d38-b139-43d0-92dd-6e4a550f2d56 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:56:08.266534 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc78d38-b139-43d0-92dd-6e4a550f2d56 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:56:08.267911 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc78d38-b139-43d0-92dd-6e4a550f2d56 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:56:08.273109 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 485/987] 10.222.0.46 () {56 vars in 1122 bytes} [Thu May 26 14:56:08 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:56:08.279481 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63970ee4-6c4f-456c-a5ac-bcb3e29c67c0 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:56:08.383870 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63970ee4-6c4f-456c-a5ac-bcb3e29c67c0 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:56:08.417766 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d37039-047b-494c-82ab-9fbd7ea7cc6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:56:08.418420 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d37039-047b-494c-82ab-9fbd7ea7cc6a None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:08.418701 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d37039-047b-494c-82ab-9fbd7ea7cc6a None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:08.418977 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d37039-047b-494c-82ab-9fbd7ea7cc6a None admin] PATH_INFO: `/v3/endpoints` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:08.419767 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d37039-047b-494c-82ab-9fbd7ea7cc6a None admin] RBAC: Adding query filter params (interface=admin, service_id=dcb32f2292d8496aa6b34425ff9f2e93, region_id=RegionOne) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:56:08.420079 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d37039-047b-494c-82ab-9fbd7ea7cc6a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:56:08.421426 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d37039-047b-494c-82ab-9fbd7ea7cc6a None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:56:08.431120 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 503/988] 10.222.0.46 () {56 vars in 1249 bytes} [Thu May 26 14:56:08 2022] GET /identity/v3/endpoints?service_id=dcb32f2292d8496aa6b34425ff9f2e93&interface=admin®ion_id=RegionOne => generated 193 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:56:08.432076 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc78d38-b139-43d0-92dd-6e4a550f2d56 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:56:08.543548 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc78d38-b139-43d0-92dd-6e4a550f2d56 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:56:08.579699 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860b7ac0-51d0-4649-9cb0-4e3982748d7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:56:08.580367 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860b7ac0-51d0-4649-9cb0-4e3982748d7d None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:08.580654 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860b7ac0-51d0-4649-9cb0-4e3982748d7d None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:08.580929 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860b7ac0-51d0-4649-9cb0-4e3982748d7d None admin] PATH_INFO: `/v3/services` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:08.581633 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860b7ac0-51d0-4649-9cb0-4e3982748d7d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:56:08.583023 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860b7ac0-51d0-4649-9cb0-4e3982748d7d None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:56:08.597868 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 486/989] 10.222.0.46 () {56 vars in 1087 bytes} [Thu May 26 14:56:08 2022] GET /identity/v3/services => generated 1727 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 14:56:08.664896 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e1f27b-6082-4224-8b5c-b20184b57144 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:08.665214 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e1f27b-6082-4224-8b5c-b20184b57144 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:08.665483 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e1f27b-6082-4224-8b5c-b20184b57144 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:08.667098 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 504/990] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 14:56:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 14:56:08.672718 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf778d76-e921-49a6-bd4c-d9cdc81be50f None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:08.673022 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf778d76-e921-49a6-bd4c-d9cdc81be50f None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:08.673283 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf778d76-e921-49a6-bd4c-d9cdc81be50f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:08.691382 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-bf778d76-e921-49a6-bd4c-d9cdc81be50f None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:56:08.692601 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-bf778d76-e921-49a6-bd4c-d9cdc81be50f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:56:08.726798 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:56:08.771580 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 487/991] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 14:56:08 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 14:56:08.796722 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e1f27b-6082-4224-8b5c-b20184b57144 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:56:08.823943 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:56:08.909789 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e1f27b-6082-4224-8b5c-b20184b57144 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:56:08.952614 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baed8a5f-c799-4155-aa93-225fbde71083 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:56:08.954340 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baed8a5f-c799-4155-aa93-225fbde71083 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:08.954340 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baed8a5f-c799-4155-aa93-225fbde71083 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:08.954340 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baed8a5f-c799-4155-aa93-225fbde71083 None admin] PATH_INFO: `/v3/services/image` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:08.954987 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baed8a5f-c799-4155-aa93-225fbde71083 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:56:08.956829 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baed8a5f-c799-4155-aa93-225fbde71083 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:56:08.973555 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-baed8a5f-c799-4155-aa93-225fbde71083 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 26 14:56:08.973555 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:56:08.973555 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 14:56:08.973555 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 14:56:08.973555 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 14:56:08.973555 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 14:56:08.973555 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 14:56:08.973555 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 14:56:08.973555 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 26 14:56:08.973555 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 26 14:56:08.973555 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 14:56:08.973555 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 26 14:56:08.973555 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:56:08.973555 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:56:08.973555 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:56:08.973555 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:56:08.973555 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:56:08.974862 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:56:08.974862 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:56:08.974862 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:56:08.974862 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:56:08.974862 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:56:08.974862 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:56:08.974862 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 14:56:08.974862 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 14:56:08.974862 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 14:56:08.974862 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 14:56:08.974862 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:56:08.974862 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:56:08.974862 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:56:08.974862 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:56:08.974862 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:56:08.974862 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:56:08.975714 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:56:08.975714 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 26 14:56:08.975714 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:56:08.975714 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 26 14:56:08.975714 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:56:08.975714 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:56:08.975714 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:56:08.975714 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 26 14:56:08.975714 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:56:08.975714 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 26 14:56:08.975714 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 14:56:08.975714 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 14:56:08.975714 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 26 14:56:08.975714 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 26 14:56:08.975714 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 505/992] 10.222.0.46 () {56 vars in 1105 bytes} [Thu May 26 14:56:08 2022] GET /identity/v3/services/image => generated 86 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:56:08.983906 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf778d76-e921-49a6-bd4c-d9cdc81be50f None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:56:09.094182 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf778d76-e921-49a6-bd4c-d9cdc81be50f None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:56:09.130067 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf53dc6-a067-4fa4-a8f2-7fabf8d884d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:56:09.130745 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf53dc6-a067-4fa4-a8f2-7fabf8d884d8 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:09.131027 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf53dc6-a067-4fa4-a8f2-7fabf8d884d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:09.131300 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf53dc6-a067-4fa4-a8f2-7fabf8d884d8 None admin] PATH_INFO: `/v3/services` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:09.132027 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf53dc6-a067-4fa4-a8f2-7fabf8d884d8 None admin] RBAC: Adding query filter params (name=image) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:56:09.132301 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf53dc6-a067-4fa4-a8f2-7fabf8d884d8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:56:09.133726 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf53dc6-a067-4fa4-a8f2-7fabf8d884d8 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:56:09.143098 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 488/993] 10.222.0.46 () {56 vars in 1108 bytes} [Thu May 26 14:56:08 2022] GET /identity/v3/services?name=image => generated 122 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:56:09.144617 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baed8a5f-c799-4155-aa93-225fbde71083 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:56:09.226726 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44014ca7-ca6d-4917-b2ff-94ec6d0d68bb None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:09.227683 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44014ca7-ca6d-4917-b2ff-94ec6d0d68bb None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:09.227859 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44014ca7-ca6d-4917-b2ff-94ec6d0d68bb None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:09.229434 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 489/994] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 14:56:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 14:56:09.233239 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:56:09.234055 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8c53ac-2afe-4cf0-b791-c806a906ac79 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:09.234345 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8c53ac-2afe-4cf0-b791-c806a906ac79 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:09.234629 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8c53ac-2afe-4cf0-b791-c806a906ac79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:09.250557 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baed8a5f-c799-4155-aa93-225fbde71083 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:56:09.250934 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-0a8c53ac-2afe-4cf0-b791-c806a906ac79 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:56:09.252121 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-0a8c53ac-2afe-4cf0-b791-c806a906ac79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:56:09.276300 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:56:09.289005 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b817b6-e37f-4d51-b8fe-a01b847fbd45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:56:09.290545 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b817b6-e37f-4d51-b8fe-a01b847fbd45 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:09.291017 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b817b6-e37f-4d51-b8fe-a01b847fbd45 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:09.291512 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b817b6-e37f-4d51-b8fe-a01b847fbd45 None admin] PATH_INFO: `/v3/services` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:09.292470 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b817b6-e37f-4d51-b8fe-a01b847fbd45 None admin] RBAC: Adding query filter params (type=image) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:56:09.292975 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b817b6-e37f-4d51-b8fe-a01b847fbd45 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:56:09.294689 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b817b6-e37f-4d51-b8fe-a01b847fbd45 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:56:09.300237 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 506/995] 10.222.0.46 () {56 vars in 1108 bytes} [Thu May 26 14:56:09 2022] GET /identity/v3/services?type=image => generated 122 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:56:09.332933 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 490/996] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 14:56:09 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 26 14:56:09.356982 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b817b6-e37f-4d51-b8fe-a01b847fbd45 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:56:09.467106 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b817b6-e37f-4d51-b8fe-a01b847fbd45 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:56:09.502918 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2eb64a-f370-4fd7-a5bd-e1ba687fc5ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:56:09.503727 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2eb64a-f370-4fd7-a5bd-e1ba687fc5ee None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:09.504272 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2eb64a-f370-4fd7-a5bd-e1ba687fc5ee None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:09.504557 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2eb64a-f370-4fd7-a5bd-e1ba687fc5ee None admin] PATH_INFO: `/v3/services/compute` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:09.505216 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2eb64a-f370-4fd7-a5bd-e1ba687fc5ee None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:56:09.506632 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2eb64a-f370-4fd7-a5bd-e1ba687fc5ee None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:56:09.519039 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-8f2eb64a-f370-4fd7-a5bd-e1ba687fc5ee None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 26 14:56:09.519039 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:56:09.519039 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 14:56:09.519039 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 14:56:09.519039 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 14:56:09.519039 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 14:56:09.519039 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 14:56:09.519039 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 14:56:09.519039 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 26 14:56:09.519039 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 26 14:56:09.519039 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 14:56:09.519039 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 26 14:56:09.519039 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:56:09.519039 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:56:09.519039 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:56:09.519039 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:56:09.519039 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:56:09.520110 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:56:09.520110 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:56:09.520110 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:56:09.520110 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:56:09.520110 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:56:09.520110 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:56:09.520110 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 14:56:09.520110 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 14:56:09.520110 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 14:56:09.520110 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 14:56:09.520110 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:56:09.520110 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:56:09.520110 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:56:09.520110 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:56:09.520110 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:56:09.520110 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:56:09.521122 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:56:09.521122 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 26 14:56:09.521122 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:56:09.521122 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 26 14:56:09.521122 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:56:09.521122 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:56:09.521122 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:56:09.521122 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 26 14:56:09.521122 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:56:09.521122 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 26 14:56:09.521122 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 14:56:09.521122 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 14:56:09.521122 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 26 14:56:09.521122 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 26 14:56:09.521122 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 507/997] 10.222.0.46 () {56 vars in 1111 bytes} [Thu May 26 14:56:09 2022] GET /identity/v3/services/compute => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:56:09.526965 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8c53ac-2afe-4cf0-b791-c806a906ac79 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:56:09.637330 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8c53ac-2afe-4cf0-b791-c806a906ac79 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:56:09.671840 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e636e6-07ba-479b-b911-0732c4822e60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:56:09.672520 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e636e6-07ba-479b-b911-0732c4822e60 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:09.672855 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e636e6-07ba-479b-b911-0732c4822e60 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:09.673141 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e636e6-07ba-479b-b911-0732c4822e60 None admin] PATH_INFO: `/v3/services` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:09.673888 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e636e6-07ba-479b-b911-0732c4822e60 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:56:09.674200 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e636e6-07ba-479b-b911-0732c4822e60 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:56:09.675585 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e636e6-07ba-479b-b911-0732c4822e60 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:56:09.685058 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 491/998] 10.222.0.46 () {56 vars in 1112 bytes} [Thu May 26 14:56:09 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:56:09.686601 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2eb64a-f370-4fd7-a5bd-e1ba687fc5ee None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:56:09.704147 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:56:09.798962 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2eb64a-f370-4fd7-a5bd-e1ba687fc5ee None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:56:09.836119 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9545b87e-a812-42f5-9b1a-40b72db0b573 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:56:09.837433 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9545b87e-a812-42f5-9b1a-40b72db0b573 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:09.837745 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9545b87e-a812-42f5-9b1a-40b72db0b573 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:09.838020 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9545b87e-a812-42f5-9b1a-40b72db0b573 None admin] PATH_INFO: `/v3/services` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:09.838730 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9545b87e-a812-42f5-9b1a-40b72db0b573 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:56:09.839038 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9545b87e-a812-42f5-9b1a-40b72db0b573 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:56:09.840502 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9545b87e-a812-42f5-9b1a-40b72db0b573 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:56:09.846089 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 508/999] 10.222.0.46 () {56 vars in 1112 bytes} [Thu May 26 14:56:09 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:56:09.851885 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e636e6-07ba-479b-b911-0732c4822e60 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:56:09.879404 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:56:09.965488 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e636e6-07ba-479b-b911-0732c4822e60 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:56:10.000101 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f2e48c-e5b9-4066-be22-3b2855bc81e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:56:10.000768 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f2e48c-e5b9-4066-be22-3b2855bc81e9 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:10.001055 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f2e48c-e5b9-4066-be22-3b2855bc81e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:10.001333 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f2e48c-e5b9-4066-be22-3b2855bc81e9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:10.002159 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f2e48c-e5b9-4066-be22-3b2855bc81e9 None admin] RBAC: Adding query filter params (interface=public, service_id=e177e366a74e4387a327c334b57a13db, region_id=RegionOne) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:56:10.002476 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f2e48c-e5b9-4066-be22-3b2855bc81e9 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:56:10.003861 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f2e48c-e5b9-4066-be22-3b2855bc81e9 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:56:10.014815 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 492/1000] 10.222.0.46 () {56 vars in 1251 bytes} [Thu May 26 14:56:09 2022] GET /identity/v3/endpoints?service_id=e177e366a74e4387a327c334b57a13db&interface=public®ion_id=RegionOne => generated 194 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:56:10.015697 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9545b87e-a812-42f5-9b1a-40b72db0b573 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:56:10.099397 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:56:10.122423 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9545b87e-a812-42f5-9b1a-40b72db0b573 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:56:10.157490 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a2b2be-8703-4067-b96c-a5693b2d6871 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:56:10.158062 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a2b2be-8703-4067-b96c-a5693b2d6871 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:10.158348 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a2b2be-8703-4067-b96c-a5693b2d6871 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:10.158626 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a2b2be-8703-4067-b96c-a5693b2d6871 None admin] PATH_INFO: `/v3/services` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:10.159315 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a2b2be-8703-4067-b96c-a5693b2d6871 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:56:10.160706 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a2b2be-8703-4067-b96c-a5693b2d6871 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:56:10.173530 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 509/1001] 10.222.0.46 () {56 vars in 1087 bytes} [Thu May 26 14:56:10 2022] GET /identity/v3/services => generated 1727 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 14:56:10.937265 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3eb86d3-49e5-4ebe-a0e4-f0c322681520 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:10.937820 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3eb86d3-49e5-4ebe-a0e4-f0c322681520 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:10.937820 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3eb86d3-49e5-4ebe-a0e4-f0c322681520 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:10.939301 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 493/1002] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 14:56:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 14:56:10.944586 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bb46ad-9210-40e7-9536-1fa094cc9b16 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:10.945010 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bb46ad-9210-40e7-9536-1fa094cc9b16 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:10.945204 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bb46ad-9210-40e7-9536-1fa094cc9b16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:10.963722 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-31bb46ad-9210-40e7-9536-1fa094cc9b16 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:56:10.964912 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-31bb46ad-9210-40e7-9536-1fa094cc9b16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:56:11.037491 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 510/1003] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 14:56:10 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 14:56:11.061844 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3eb86d3-49e5-4ebe-a0e4-f0c322681520 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:56:11.160905 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:56:11.176226 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3eb86d3-49e5-4ebe-a0e4-f0c322681520 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:56:11.212991 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f36b57-c5fe-4ebc-8da2-b55c6533f63b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:56:11.213750 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f36b57-c5fe-4ebc-8da2-b55c6533f63b None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:11.214056 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f36b57-c5fe-4ebc-8da2-b55c6533f63b None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:11.214339 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f36b57-c5fe-4ebc-8da2-b55c6533f63b None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:11.215051 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f36b57-c5fe-4ebc-8da2-b55c6533f63b None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:56:11.216572 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f36b57-c5fe-4ebc-8da2-b55c6533f63b None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:56:11.229677 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-28f36b57-c5fe-4ebc-8da2-b55c6533f63b None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 26 14:56:11.229677 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:56:11.229677 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 14:56:11.229677 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 14:56:11.229677 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 14:56:11.229677 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 14:56:11.229677 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 14:56:11.229677 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 14:56:11.229677 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 26 14:56:11.229677 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 26 14:56:11.229677 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 14:56:11.229677 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 26 14:56:11.229677 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:56:11.229677 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:56:11.229677 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:56:11.229677 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:56:11.229677 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:56:11.230747 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:56:11.230747 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:56:11.230747 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:56:11.230747 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:56:11.230747 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:56:11.230747 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:56:11.230747 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 14:56:11.230747 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 14:56:11.230747 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 14:56:11.230747 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 14:56:11.230747 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:56:11.230747 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:56:11.230747 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:56:11.230747 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:56:11.230747 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:56:11.230747 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:56:11.231599 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:56:11.231599 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 26 14:56:11.231599 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:56:11.231599 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 26 14:56:11.231599 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:56:11.231599 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:56:11.231599 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:56:11.231599 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 26 14:56:11.231599 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:56:11.231599 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 26 14:56:11.231599 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 14:56:11.231599 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 14:56:11.231599 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 26 14:56:11.231599 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 26 14:56:11.233790 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 494/1004] 10.222.0.46 () {56 vars in 1126 bytes} [Thu May 26 14:56:11 2022] GET /identity/v3/services/object-store => generated 93 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:56:11.236837 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bb46ad-9210-40e7-9536-1fa094cc9b16 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:56:11.309103 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:56:11.340883 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bb46ad-9210-40e7-9536-1fa094cc9b16 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:56:11.374940 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33080f61-c168-4612-ab08-12de7e3320a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:56:11.375616 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33080f61-c168-4612-ab08-12de7e3320a8 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:11.375900 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33080f61-c168-4612-ab08-12de7e3320a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:11.376175 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33080f61-c168-4612-ab08-12de7e3320a8 None admin] PATH_INFO: `/v3/services` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:11.376888 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33080f61-c168-4612-ab08-12de7e3320a8 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:56:11.377194 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33080f61-c168-4612-ab08-12de7e3320a8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:56:11.378608 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33080f61-c168-4612-ab08-12de7e3320a8 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:56:11.385110 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 511/1005] 10.222.0.46 () {56 vars in 1122 bytes} [Thu May 26 14:56:11 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:56:11.390798 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f36b57-c5fe-4ebc-8da2-b55c6533f63b None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:56:11.496815 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f36b57-c5fe-4ebc-8da2-b55c6533f63b None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:56:11.531747 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0cf7d01-ba52-430c-a1c3-005c6d9a1994 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:56:11.532417 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cf7d01-ba52-430c-a1c3-005c6d9a1994 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:11.532700 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cf7d01-ba52-430c-a1c3-005c6d9a1994 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:11.532978 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0cf7d01-ba52-430c-a1c3-005c6d9a1994 None admin] PATH_INFO: `/v3/services` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:11.533713 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0cf7d01-ba52-430c-a1c3-005c6d9a1994 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:56:11.534022 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0cf7d01-ba52-430c-a1c3-005c6d9a1994 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:56:11.535397 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0cf7d01-ba52-430c-a1c3-005c6d9a1994 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:56:11.545248 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 495/1006] 10.222.0.46 () {56 vars in 1122 bytes} [Thu May 26 14:56:11 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:56:11.546813 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33080f61-c168-4612-ab08-12de7e3320a8 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:56:11.583229 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c83b59-1271-4c7b-a478-89523a40e81a None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:11.583556 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c83b59-1271-4c7b-a478-89523a40e81a None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:11.583837 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c83b59-1271-4c7b-a478-89523a40e81a None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:11.585206 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 496/1007] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 14:56:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 14:56:11.590244 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede6e050-0b8d-415a-b388-cd22c9eb8fdb None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:11.590536 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede6e050-0b8d-415a-b388-cd22c9eb8fdb None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:11.590816 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede6e050-0b8d-415a-b388-cd22c9eb8fdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:11.610662 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-ede6e050-0b8d-415a-b388-cd22c9eb8fdb None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:56:11.611992 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-ede6e050-0b8d-415a-b388-cd22c9eb8fdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:56:11.624770 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:56:11.658376 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33080f61-c168-4612-ab08-12de7e3320a8 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:56:11.694589 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 497/1008] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 14:56:11 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 26 14:56:11.696217 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd318760-0d80-4f4c-be40-230976bbed79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:56:11.696883 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd318760-0d80-4f4c-be40-230976bbed79 None admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:11.697165 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd318760-0d80-4f4c-be40-230976bbed79 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:11.697436 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd318760-0d80-4f4c-be40-230976bbed79 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:11.698198 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd318760-0d80-4f4c-be40-230976bbed79 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:56:11.699655 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd318760-0d80-4f4c-be40-230976bbed79 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:56:11.704260 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:56:11.720344 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede6e050-0b8d-415a-b388-cd22c9eb8fdb None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:56:11.721371 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 512/1009] 10.222.0.46 () {60 vars in 1144 bytes} [Thu May 26 14:56:11 2022] POST /identity/v3/endpoints => generated 325 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 14:56:11.845298 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede6e050-0b8d-415a-b388-cd22c9eb8fdb None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:56:11.895576 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1213cf-6ade-453e-ab65-9979d2b57722 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:56:11.896262 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1213cf-6ade-453e-ab65-9979d2b57722 None admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:11.897079 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1213cf-6ade-453e-ab65-9979d2b57722 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:11.897079 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1213cf-6ade-453e-ab65-9979d2b57722 None admin] PATH_INFO: `/v3/services` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:11.897642 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1213cf-6ade-453e-ab65-9979d2b57722 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:56:11.899065 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1213cf-6ade-453e-ab65-9979d2b57722 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:56:11.917489 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 498/1010] 10.222.0.46 () {60 vars in 1141 bytes} [Thu May 26 14:56:11 2022] POST /identity/v3/services => generated 242 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 14:56:12.601997 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0125ef7-7bc4-4ca7-bd90-653f18dfd08d None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:12.602630 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0125ef7-7bc4-4ca7-bd90-653f18dfd08d None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:12.602630 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0125ef7-7bc4-4ca7-bd90-653f18dfd08d None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:12.603942 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 513/1011] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 14:56:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 14:56:12.611067 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8844694e-3253-4da1-8de2-6fb5f6f487ff None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:12.611572 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8844694e-3253-4da1-8de2-6fb5f6f487ff None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:12.611658 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8844694e-3253-4da1-8de2-6fb5f6f487ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:12.617848 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:56:12.635232 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-8844694e-3253-4da1-8de2-6fb5f6f487ff None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:56:12.636418 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-8844694e-3253-4da1-8de2-6fb5f6f487ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:56:12.709510 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 499/1012] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 14:56:12 2022] POST /identity/v3/auth/tokens => generated 1803 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 14:56:12.732262 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0125ef7-7bc4-4ca7-bd90-653f18dfd08d None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:56:12.838356 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0125ef7-7bc4-4ca7-bd90-653f18dfd08d None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:56:12.872821 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a7fd23-13bc-47ff-987e-2d4631810b7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:56:12.873715 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a7fd23-13bc-47ff-987e-2d4631810b7c None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:12.874011 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a7fd23-13bc-47ff-987e-2d4631810b7c None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:12.874288 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a7fd23-13bc-47ff-987e-2d4631810b7c None admin] PATH_INFO: `/v3/services/compute` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:12.874951 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a7fd23-13bc-47ff-987e-2d4631810b7c None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:56:12.876310 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a7fd23-13bc-47ff-987e-2d4631810b7c None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:56:12.889532 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-34a7fd23-13bc-47ff-987e-2d4631810b7c None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 26 14:56:12.889532 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:56:12.889532 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 14:56:12.889532 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 14:56:12.889532 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 14:56:12.889532 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 14:56:12.889532 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 14:56:12.889532 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 14:56:12.889532 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 26 14:56:12.889532 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 26 14:56:12.889532 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 14:56:12.889532 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 26 14:56:12.889532 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:56:12.889532 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:56:12.889532 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:56:12.889532 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:56:12.889532 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:56:12.890691 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:56:12.890691 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:56:12.890691 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:56:12.890691 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:56:12.890691 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:56:12.890691 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:56:12.890691 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 14:56:12.890691 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 14:56:12.890691 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 14:56:12.890691 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 14:56:12.890691 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:56:12.890691 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:56:12.890691 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:56:12.890691 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:56:12.890691 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:56:12.890691 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:56:12.891637 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:56:12.891637 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 26 14:56:12.891637 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:56:12.891637 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 26 14:56:12.891637 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:56:12.891637 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:56:12.891637 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:56:12.891637 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 26 14:56:12.891637 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:56:12.891637 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 26 14:56:12.891637 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 14:56:12.891637 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 14:56:12.891637 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 26 14:56:12.891637 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 26 14:56:12.893548 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 514/1013] 10.222.0.46 () {56 vars in 1111 bytes} [Thu May 26 14:56:12 2022] GET /identity/v3/services/compute => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:56:12.897596 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8844694e-3253-4da1-8de2-6fb5f6f487ff None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:56:13.000755 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8844694e-3253-4da1-8de2-6fb5f6f487ff None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:56:13.028825 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:56:13.037985 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9809baf-aa39-4475-811f-3d1929802d96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:56:13.038661 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9809baf-aa39-4475-811f-3d1929802d96 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:13.038945 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9809baf-aa39-4475-811f-3d1929802d96 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:13.039222 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9809baf-aa39-4475-811f-3d1929802d96 None admin] PATH_INFO: `/v3/services` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:13.039972 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9809baf-aa39-4475-811f-3d1929802d96 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:56:13.040280 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9809baf-aa39-4475-811f-3d1929802d96 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:56:13.041697 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9809baf-aa39-4475-811f-3d1929802d96 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:56:13.046862 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 500/1014] 10.222.0.46 () {56 vars in 1112 bytes} [Thu May 26 14:56:12 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:56:13.052653 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a7fd23-13bc-47ff-987e-2d4631810b7c None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:56:13.116690 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:56:13.162900 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a7fd23-13bc-47ff-987e-2d4631810b7c None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:56:13.199225 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716937df-d9ba-4877-8aa5-24da94027ec9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:56:13.199915 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716937df-d9ba-4877-8aa5-24da94027ec9 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:13.200200 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716937df-d9ba-4877-8aa5-24da94027ec9 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:13.200487 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716937df-d9ba-4877-8aa5-24da94027ec9 None admin] PATH_INFO: `/v3/services` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:13.201204 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716937df-d9ba-4877-8aa5-24da94027ec9 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:56:13.201539 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716937df-d9ba-4877-8aa5-24da94027ec9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:56:13.202934 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716937df-d9ba-4877-8aa5-24da94027ec9 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:56:13.214395 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 515/1015] 10.222.0.46 () {56 vars in 1112 bytes} [Thu May 26 14:56:13 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:56:13.216088 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9809baf-aa39-4475-811f-3d1929802d96 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:56:13.322562 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9809baf-aa39-4475-811f-3d1929802d96 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:56:13.356292 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91258559-9245-4bf6-bac5-6203df22095e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:56:13.356951 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91258559-9245-4bf6-bac5-6203df22095e None admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:13.357233 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91258559-9245-4bf6-bac5-6203df22095e None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:13.357534 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91258559-9245-4bf6-bac5-6203df22095e None admin] PATH_INFO: `/v3/endpoints` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:13.358277 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91258559-9245-4bf6-bac5-6203df22095e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:56:13.359657 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91258559-9245-4bf6-bac5-6203df22095e None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:56:13.381487 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 501/1016] 10.222.0.46 () {60 vars in 1144 bytes} [Thu May 26 14:56:13 2022] POST /identity/v3/endpoints => generated 334 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 14:56:14.013229 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670cc1d6-37d9-4168-98d8-341acf5cc8b2 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:14.013793 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670cc1d6-37d9-4168-98d8-341acf5cc8b2 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:14.013793 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670cc1d6-37d9-4168-98d8-341acf5cc8b2 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:14.015129 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 516/1017] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 14:56:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 14:56:14.021393 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffddad7e-c558-463c-9da1-92874178e7fe None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:14.021962 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffddad7e-c558-463c-9da1-92874178e7fe None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:14.021962 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffddad7e-c558-463c-9da1-92874178e7fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:14.041212 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-ffddad7e-c558-463c-9da1-92874178e7fe None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:56:14.042472 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-ffddad7e-c558-463c-9da1-92874178e7fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:56:14.089106 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:56:14.121601 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 502/1018] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 14:56:14 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 14:56:14.145507 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670cc1d6-37d9-4168-98d8-341acf5cc8b2 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:56:14.214451 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1a82a9-8ef1-424b-9597-0ad566a968ec None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:14.214820 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1a82a9-8ef1-424b-9597-0ad566a968ec None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:14.215105 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1a82a9-8ef1-424b-9597-0ad566a968ec None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:14.216745 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 503/1019] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 14:56:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 14:56:14.223192 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bc314f-7855-4150-932b-5e7f3d95df53 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:14.223478 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bc314f-7855-4150-932b-5e7f3d95df53 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:14.223756 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bc314f-7855-4150-932b-5e7f3d95df53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:14.246371 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-75bc314f-7855-4150-932b-5e7f3d95df53 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:56:14.248274 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-75bc314f-7855-4150-932b-5e7f3d95df53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:56:14.253624 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-670cc1d6-37d9-4168-98d8-341acf5cc8b2 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:56:14.280463 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:56:14.292079 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677a9c22-37ed-433e-b093-efa6ce31dd0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:56:14.292882 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677a9c22-37ed-433e-b093-efa6ce31dd0d None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:14.293161 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677a9c22-37ed-433e-b093-efa6ce31dd0d None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:14.293430 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-677a9c22-37ed-433e-b093-efa6ce31dd0d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:14.294708 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677a9c22-37ed-433e-b093-efa6ce31dd0d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:56:14.296136 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-677a9c22-37ed-433e-b093-efa6ce31dd0d None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:56:14.301785 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 517/1020] 10.222.0.46 () {56 vars in 1108 bytes} [Thu May 26 14:56:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:56:14.303436 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677a9c22-37ed-433e-b093-efa6ce31dd0d None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:56:14.335333 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 504/1021] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 14:56:14 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 14:56:14.358367 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bc314f-7855-4150-932b-5e7f3d95df53 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:56:14.421786 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-677a9c22-37ed-433e-b093-efa6ce31dd0d None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:56:14.458151 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e190934-9ef1-475e-a5d3-11d9207f1de1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:56:14.458802 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e190934-9ef1-475e-a5d3-11d9207f1de1 None admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:14.459089 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e190934-9ef1-475e-a5d3-11d9207f1de1 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:14.459363 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e190934-9ef1-475e-a5d3-11d9207f1de1 None admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:14.460074 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e190934-9ef1-475e-a5d3-11d9207f1de1 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:56:14.462087 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e190934-9ef1-475e-a5d3-11d9207f1de1 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:56:14.463548 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bc314f-7855-4150-932b-5e7f3d95df53 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:56:14.465975 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:56:14.490478 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 518/1022] 10.222.0.46 () {60 vars in 1141 bytes} [Thu May 26 14:56:14 2022] POST /identity/v3/projects => generated 311 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 14:56:14.505552 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5318dd99-8d6e-4c17-b7e8-d4e0d40c93a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:56:14.506312 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5318dd99-8d6e-4c17-b7e8-d4e0d40c93a9 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:14.506600 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5318dd99-8d6e-4c17-b7e8-d4e0d40c93a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:14.506879 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5318dd99-8d6e-4c17-b7e8-d4e0d40c93a9 None admin] PATH_INFO: `/v3/services/image` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:14.507554 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5318dd99-8d6e-4c17-b7e8-d4e0d40c93a9 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:56:14.509603 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5318dd99-8d6e-4c17-b7e8-d4e0d40c93a9 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:56:14.522603 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-5318dd99-8d6e-4c17-b7e8-d4e0d40c93a9 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 26 14:56:14.522603 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:56:14.522603 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 14:56:14.522603 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 14:56:14.522603 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 14:56:14.522603 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 14:56:14.522603 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 14:56:14.522603 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 14:56:14.522603 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 26 14:56:14.522603 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 26 14:56:14.522603 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 14:56:14.522603 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 26 14:56:14.522603 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:56:14.522603 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:56:14.522603 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:56:14.522603 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:56:14.522603 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:56:14.523746 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:56:14.523746 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:56:14.523746 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:56:14.523746 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:56:14.523746 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:56:14.523746 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:56:14.523746 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 14:56:14.523746 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 14:56:14.523746 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 14:56:14.523746 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 14:56:14.523746 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:56:14.523746 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:56:14.523746 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:56:14.523746 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:56:14.523746 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:56:14.523746 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:56:14.524627 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:56:14.524627 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 26 14:56:14.524627 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:56:14.524627 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 26 14:56:14.524627 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:56:14.524627 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:56:14.524627 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:56:14.524627 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 26 14:56:14.524627 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:56:14.524627 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 26 14:56:14.524627 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 14:56:14.524627 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 14:56:14.524627 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 26 14:56:14.524627 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 26 14:56:14.524627 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 505/1023] 10.222.0.46 () {56 vars in 1105 bytes} [Thu May 26 14:56:14 2022] GET /identity/v3/services/image => generated 86 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 14:56:14.530363 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e190934-9ef1-475e-a5d3-11d9207f1de1 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:56:14.585928 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:56:14.659913 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e190934-9ef1-475e-a5d3-11d9207f1de1 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:56:14.699147 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feaf2585-1165-4a3a-9ebb-0407e7e987a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:56:14.699824 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feaf2585-1165-4a3a-9ebb-0407e7e987a2 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:14.700179 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feaf2585-1165-4a3a-9ebb-0407e7e987a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:14.700460 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feaf2585-1165-4a3a-9ebb-0407e7e987a2 None admin] PATH_INFO: `/v3/services` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:14.701173 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feaf2585-1165-4a3a-9ebb-0407e7e987a2 None admin] RBAC: Adding query filter params (name=image) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:56:14.701501 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feaf2585-1165-4a3a-9ebb-0407e7e987a2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:56:14.702869 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feaf2585-1165-4a3a-9ebb-0407e7e987a2 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:56:14.708769 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 519/1024] 10.222.0.46 () {56 vars in 1108 bytes} [Thu May 26 14:56:14 2022] GET /identity/v3/services?name=image => generated 122 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:56:14.714542 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5318dd99-8d6e-4c17-b7e8-d4e0d40c93a9 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:56:14.819764 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5318dd99-8d6e-4c17-b7e8-d4e0d40c93a9 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:56:14.868341 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae3e6ff-b904-4bc7-beca-dc9ee92efc85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:56:14.868921 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae3e6ff-b904-4bc7-beca-dc9ee92efc85 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:14.869200 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae3e6ff-b904-4bc7-beca-dc9ee92efc85 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:14.872590 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae3e6ff-b904-4bc7-beca-dc9ee92efc85 None admin] PATH_INFO: `/v3/services` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:14.873325 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae3e6ff-b904-4bc7-beca-dc9ee92efc85 None admin] RBAC: Adding query filter params (type=image) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:56:14.873659 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae3e6ff-b904-4bc7-beca-dc9ee92efc85 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:56:14.875096 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae3e6ff-b904-4bc7-beca-dc9ee92efc85 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:56:14.885951 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 506/1025] 10.222.0.46 () {56 vars in 1108 bytes} [Thu May 26 14:56:14 2022] GET /identity/v3/services?type=image => generated 350 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:56:14.887015 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feaf2585-1165-4a3a-9ebb-0407e7e987a2 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:56:14.986661 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feaf2585-1165-4a3a-9ebb-0407e7e987a2 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:56:14.996285 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:56:15.022250 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185813c5-8a0d-431b-930b-19df321fcb29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:56:15.023014 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185813c5-8a0d-431b-930b-19df321fcb29 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:15.023200 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185813c5-8a0d-431b-930b-19df321fcb29 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:15.023458 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185813c5-8a0d-431b-930b-19df321fcb29 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:15.024264 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185813c5-8a0d-431b-930b-19df321fcb29 None admin] RBAC: Adding query filter params (interface=public, service_id=248f71020e084230b7331f0c8444ef46, region_id=RegionOne) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:56:15.024572 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185813c5-8a0d-431b-930b-19df321fcb29 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:56:15.025960 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185813c5-8a0d-431b-930b-19df321fcb29 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:56:15.035462 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 520/1026] 10.222.0.46 () {56 vars in 1251 bytes} [Thu May 26 14:56:14 2022] GET /identity/v3/endpoints?service_id=248f71020e084230b7331f0c8444ef46&interface=public®ion_id=RegionOne => generated 194 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 14:56:15.036629 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae3e6ff-b904-4bc7-beca-dc9ee92efc85 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:56:15.070011 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:56:15.144050 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae3e6ff-b904-4bc7-beca-dc9ee92efc85 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:56:15.180911 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443c6fdf-6525-417c-9c46-f119c3310cf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:56:15.181617 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443c6fdf-6525-417c-9c46-f119c3310cf2 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:15.181921 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443c6fdf-6525-417c-9c46-f119c3310cf2 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:15.182195 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443c6fdf-6525-417c-9c46-f119c3310cf2 None admin] PATH_INFO: `/v3/services` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:15.182846 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443c6fdf-6525-417c-9c46-f119c3310cf2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:56:15.184274 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443c6fdf-6525-417c-9c46-f119c3310cf2 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:56:15.193754 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 507/1027] 10.222.0.46 () {56 vars in 1087 bytes} [Thu May 26 14:56:15 2022] GET /identity/v3/services => generated 1957 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 14:56:15.757832 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d37d0d-6ea9-48fa-8b69-ac3f8c8a7a88 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:15.758536 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d37d0d-6ea9-48fa-8b69-ac3f8c8a7a88 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:15.758868 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d37d0d-6ea9-48fa-8b69-ac3f8c8a7a88 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:15.760202 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 521/1028] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 14:56:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 14:56:15.765610 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9419d96-39a2-4968-b910-2c386970dc44 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:15.765901 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9419d96-39a2-4968-b910-2c386970dc44 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:15.766172 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9419d96-39a2-4968-b910-2c386970dc44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:15.784172 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-a9419d96-39a2-4968-b910-2c386970dc44 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:56:15.785361 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-a9419d96-39a2-4968-b910-2c386970dc44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:56:15.853075 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 508/1029] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 14:56:15 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 14:56:15.875674 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d37d0d-6ea9-48fa-8b69-ac3f8c8a7a88 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:56:15.981515 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d37d0d-6ea9-48fa-8b69-ac3f8c8a7a88 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:56:16.032735 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34387c7-4c65-45b7-b199-9a98234bd343 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:56:16.033779 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34387c7-4c65-45b7-b199-9a98234bd343 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:16.033970 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34387c7-4c65-45b7-b199-9a98234bd343 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:16.034316 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34387c7-4c65-45b7-b199-9a98234bd343 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:16.046437 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34387c7-4c65-45b7-b199-9a98234bd343 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 14:56:16.055431 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-a34387c7-4c65-45b7-b199-9a98234bd343 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 26 14:56:16.055431 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:56:16.055431 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:56:16.055431 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:56:16.055431 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:56:16.055431 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:56:16.055431 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:56:16.055431 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:56:16.055431 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:56:16.055431 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:56:16.055431 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:56:16.055431 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:56:16.055431 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 14:56:16.055431 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 14:56:16.055431 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 14:56:16.055431 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 14:56:16.056806 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 14:56:16.056806 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 14:56:16.056806 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 14:56:16.056806 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 14:56:16.056806 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:56:16.056806 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:56:16.056806 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:56:16.056806 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:56:16.056806 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:56:16.056806 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:56:16.056806 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:56:16.056806 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 26 14:56:16.056806 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:56:16.056806 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 26 14:56:16.056806 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:56:16.056806 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:56:16.057741 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:56:16.057741 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 26 14:56:16.057741 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:56:16.057741 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 26 14:56:16.057741 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 14:56:16.057741 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 14:56:16.057741 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 14:56:16.057741 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 14:56:16.057741 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 14:56:16.057741 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 14:56:16.057741 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 26 14:56:16.057741 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 26 14:56:16.057741 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 522/1030] 10.222.0.46 () {56 vars in 1120 bytes} [Thu May 26 14:56:15 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:56:16.067833 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9419d96-39a2-4968-b910-2c386970dc44 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:56:16.099214 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:56:16.197639 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9419d96-39a2-4968-b910-2c386970dc44 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:56:16.233940 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ededf5d9-3c43-46af-a3cc-a0cd98d5ff3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:56:16.234908 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ededf5d9-3c43-46af-a3cc-a0cd98d5ff3c None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:16.235327 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ededf5d9-3c43-46af-a3cc-a0cd98d5ff3c None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:16.235606 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ededf5d9-3c43-46af-a3cc-a0cd98d5ff3c None admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:16.236431 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ededf5d9-3c43-46af-a3cc-a0cd98d5ff3c None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:56:16.236735 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ededf5d9-3c43-46af-a3cc-a0cd98d5ff3c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:56:16.238180 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ededf5d9-3c43-46af-a3cc-a0cd98d5ff3c None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:56:16.247361 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 509/1031] 10.222.0.46 () {56 vars in 1115 bytes} [Thu May 26 14:56:16 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:56:16.252733 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34387c7-4c65-45b7-b199-9a98234bd343 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:56:16.287972 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:56:16.360832 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34387c7-4c65-45b7-b199-9a98234bd343 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:56:16.395452 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424b5208-b002-43e9-86ab-86d5319a311d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:56:16.396248 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424b5208-b002-43e9-86ab-86d5319a311d None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:16.396526 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424b5208-b002-43e9-86ab-86d5319a311d None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:16.396851 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424b5208-b002-43e9-86ab-86d5319a311d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:16.408435 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424b5208-b002-43e9-86ab-86d5319a311d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:56:16.410097 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424b5208-b002-43e9-86ab-86d5319a311d None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:56:16.427372 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-424b5208-b002-43e9-86ab-86d5319a311d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 14:56:16.427372 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:56:16.427372 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:56:16.427372 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:56:16.427372 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:56:16.427372 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:56:16.427372 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:56:16.427372 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:56:16.427372 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:56:16.427372 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:56:16.427372 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:56:16.427372 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:56:16.427372 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 14:56:16.427372 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 14:56:16.427372 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 14:56:16.428428 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 14:56:16.428428 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:56:16.428428 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:56:16.428428 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:56:16.428428 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:56:16.428428 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:56:16.428428 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:56:16.428428 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:56:16.428428 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 26 14:56:16.428428 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:56:16.428428 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 26 14:56:16.428428 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:56:16.428428 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:56:16.428428 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:56:16.428428 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 26 14:56:16.428428 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:56:16.428428 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 26 14:56:16.429353 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 14:56:16.429353 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 14:56:16.429353 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 14:56:16.429353 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 26 14:56:16.430487 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 523/1032] 10.222.0.46 () {56 vars in 1108 bytes} [Thu May 26 14:56:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:56:16.435120 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ededf5d9-3c43-46af-a3cc-a0cd98d5ff3c None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:56:16.551032 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:56:16.564672 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ededf5d9-3c43-46af-a3cc-a0cd98d5ff3c None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:56:16.599800 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4a9378-036e-4ae0-b554-ddc6b494aa6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:56:16.600462 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4a9378-036e-4ae0-b554-ddc6b494aa6f None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:16.600743 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4a9378-036e-4ae0-b554-ddc6b494aa6f None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:16.601019 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4a9378-036e-4ae0-b554-ddc6b494aa6f None admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:16.601815 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4a9378-036e-4ae0-b554-ddc6b494aa6f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:56:16.602124 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4a9378-036e-4ae0-b554-ddc6b494aa6f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:56:16.603490 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4a9378-036e-4ae0-b554-ddc6b494aa6f None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:56:16.624828 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 510/1033] 10.222.0.46 () {56 vars in 1109 bytes} [Thu May 26 14:56:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:56:16.626692 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424b5208-b002-43e9-86ab-86d5319a311d None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:56:16.724449 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:56:16.741492 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424b5208-b002-43e9-86ab-86d5319a311d None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:56:16.777666 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947eacca-a3f2-4eb7-bc85-6ccc1721f518 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:56:16.778476 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947eacca-a3f2-4eb7-bc85-6ccc1721f518 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:16.778760 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947eacca-a3f2-4eb7-bc85-6ccc1721f518 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:16.779037 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947eacca-a3f2-4eb7-bc85-6ccc1721f518 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:16.787060 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947eacca-a3f2-4eb7-bc85-6ccc1721f518 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:56:16.788513 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947eacca-a3f2-4eb7-bc85-6ccc1721f518 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:56:16.803053 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-947eacca-a3f2-4eb7-bc85-6ccc1721f518 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 26 14:56:16.803053 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:56:16.803053 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:56:16.803053 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:56:16.803053 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:56:16.803053 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:56:16.803053 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:56:16.803053 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:56:16.803053 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:56:16.803053 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:56:16.803053 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:56:16.803053 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:56:16.803053 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 14:56:16.803053 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 14:56:16.803053 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 14:56:16.803053 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 14:56:16.804056 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:56:16.804056 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:56:16.804056 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 14:56:16.804056 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 14:56:16.804056 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 14:56:16.804056 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 14:56:16.804056 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:56:16.804056 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:56:16.804056 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:56:16.804056 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:56:16.804056 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:56:16.804056 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 26 14:56:16.804056 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:56:16.804056 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 26 14:56:16.804056 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:56:16.804056 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:56:16.804056 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:56:16.804056 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 26 14:56:16.810659 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:56:16.810659 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 26 14:56:16.810659 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 14:56:16.810659 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 14:56:16.810659 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 14:56:16.810659 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 14:56:16.810659 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 14:56:16.810659 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 14:56:16.810659 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 26 14:56:16.810659 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 26 14:56:16.810659 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 524/1034] 10.222.0.46 () {56 vars in 1093 bytes} [Thu May 26 14:56:16 2022] GET /identity/v3/users/nova => generated 82 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:56:16.811318 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4a9378-036e-4ae0-b554-ddc6b494aa6f None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:56:16.865474 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6308c0e-df67-4f29-a526-895dea6b4efd None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:16.865794 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6308c0e-df67-4f29-a526-895dea6b4efd None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:16.866065 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6308c0e-df67-4f29-a526-895dea6b4efd None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:16.868119 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 525/1035] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 14:56:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 14:56:16.873352 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f047f246-12ea-48b4-9b1c-c8976910ac7b None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:16.873677 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f047f246-12ea-48b4-9b1c-c8976910ac7b None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:16.873961 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f047f246-12ea-48b4-9b1c-c8976910ac7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:16.893238 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-f047f246-12ea-48b4-9b1c-c8976910ac7b None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:56:16.894480 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-f047f246-12ea-48b4-9b1c-c8976910ac7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:56:16.925019 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4a9378-036e-4ae0-b554-ddc6b494aa6f None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:56:16.965337 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 526/1036] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 14:56:16 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 26 14:56:16.967044 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2027af00-b40d-4fa0-8993-03fa92ba0675 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:56:16.967729 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2027af00-b40d-4fa0-8993-03fa92ba0675 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:16.968011 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2027af00-b40d-4fa0-8993-03fa92ba0675 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:16.968287 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2027af00-b40d-4fa0-8993-03fa92ba0675 None admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:16.969137 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2027af00-b40d-4fa0-8993-03fa92ba0675 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:56:16.969477 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2027af00-b40d-4fa0-8993-03fa92ba0675 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:56:16.970992 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2027af00-b40d-4fa0-8993-03fa92ba0675 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:56:16.994101 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 511/1037] 10.222.0.46 () {56 vars in 1133 bytes} [Thu May 26 14:56:16 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:56:16.995352 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f047f246-12ea-48b4-9b1c-c8976910ac7b None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:56:16.995469 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2027af00-b40d-4fa0-8993-03fa92ba0675 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:56:17.004534 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:56:17.079433 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:56:17.101478 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2027af00-b40d-4fa0-8993-03fa92ba0675 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:56:17.109045 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f047f246-12ea-48b4-9b1c-c8976910ac7b None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:56:17.139774 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376e2f0d-4fb3-451e-8c43-4b5f6fb79f5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:56:17.140465 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376e2f0d-4fb3-451e-8c43-4b5f6fb79f5c None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:17.140748 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376e2f0d-4fb3-451e-8c43-4b5f6fb79f5c None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:17.141019 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376e2f0d-4fb3-451e-8c43-4b5f6fb79f5c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:17.148449 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3669f3d-4315-4482-ab6b-5925841f0fd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:56:17.149564 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3669f3d-4315-4482-ab6b-5925841f0fd4 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:17.150076 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3669f3d-4315-4482-ab6b-5925841f0fd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:17.150350 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3669f3d-4315-4482-ab6b-5925841f0fd4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:17.151548 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3669f3d-4315-4482-ab6b-5925841f0fd4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:56:17.153024 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3669f3d-4315-4482-ab6b-5925841f0fd4 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:56:17.155015 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 527/1038] 10.222.0.46 () {56 vars in 1108 bytes} [Thu May 26 14:56:16 2022] GET /identity/v3/domains/default => generated 199 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 14:56:17.160533 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3669f3d-4315-4482-ab6b-5925841f0fd4 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:56:17.224499 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376e2f0d-4fb3-451e-8c43-4b5f6fb79f5c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:56:17.226260 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376e2f0d-4fb3-451e-8c43-4b5f6fb79f5c None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:56:17.237277 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:56:17.270057 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3669f3d-4315-4482-ab6b-5925841f0fd4 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:56:17.310089 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9db1a3-4895-418e-b881-ca482b8a4813 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:56:17.311100 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9db1a3-4895-418e-b881-ca482b8a4813 None admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:17.311590 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9db1a3-4895-418e-b881-ca482b8a4813 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:17.312047 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9db1a3-4895-418e-b881-ca482b8a4813 None admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:17.312932 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9db1a3-4895-418e-b881-ca482b8a4813 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:56:17.314903 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9db1a3-4895-418e-b881-ca482b8a4813 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:56:17.336095 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:56:17.346093 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 528/1039] 10.222.0.46 () {60 vars in 1132 bytes} [Thu May 26 14:56:17 2022] POST /identity/v3/users => generated 285 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 14:56:17.363845 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 512/1040] 10.222.0.46 () {58 vars in 1282 bytes} [Thu May 26 14:56:16 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 26 14:56:17.370542 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9db1a3-4895-418e-b881-ca482b8a4813 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:56:17.498642 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9db1a3-4895-418e-b881-ca482b8a4813 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:56:17.537870 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af11a4a2-5954-4bc6-acf5-63ead6d22eb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:56:17.538686 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af11a4a2-5954-4bc6-acf5-63ead6d22eb8 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:17.538968 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af11a4a2-5954-4bc6-acf5-63ead6d22eb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:17.539830 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af11a4a2-5954-4bc6-acf5-63ead6d22eb8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:17.551769 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af11a4a2-5954-4bc6-acf5-63ead6d22eb8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:56:17.553210 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af11a4a2-5954-4bc6-acf5-63ead6d22eb8 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:56:17.571741 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-af11a4a2-5954-4bc6-acf5-63ead6d22eb8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 14:56:17.571741 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:56:17.571741 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:56:17.571741 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:56:17.571741 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:56:17.571741 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:56:17.571741 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:56:17.571741 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:56:17.571741 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:56:17.571741 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:56:17.571741 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:56:17.571741 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:56:17.571741 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 14:56:17.571741 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 14:56:17.571741 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 14:56:17.572898 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 14:56:17.572898 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:56:17.572898 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:56:17.572898 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:56:17.572898 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:56:17.572898 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:56:17.572898 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:56:17.572898 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:56:17.572898 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 26 14:56:17.572898 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:56:17.572898 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 26 14:56:17.572898 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:56:17.572898 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:56:17.572898 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:56:17.572898 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 26 14:56:17.572898 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:56:17.572898 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 26 14:56:17.573967 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 14:56:17.573967 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 14:56:17.573967 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 14:56:17.573967 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 26 14:56:17.573967 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 529/1041] 10.222.0.46 () {56 vars in 1108 bytes} [Thu May 26 14:56:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:56:17.580076 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376e2f0d-4fb3-451e-8c43-4b5f6fb79f5c None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:56:17.695373 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376e2f0d-4fb3-451e-8c43-4b5f6fb79f5c None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:56:17.703644 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:56:17.734141 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc8dbfc-b2c1-45bf-8ae9-9ea1b04bfedc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:56:17.735119 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc8dbfc-b2c1-45bf-8ae9-9ea1b04bfedc None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:17.735306 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc8dbfc-b2c1-45bf-8ae9-9ea1b04bfedc None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:17.735611 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc8dbfc-b2c1-45bf-8ae9-9ea1b04bfedc None admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:17.736544 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc8dbfc-b2c1-45bf-8ae9-9ea1b04bfedc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:56:17.736853 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc8dbfc-b2c1-45bf-8ae9-9ea1b04bfedc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:56:17.738264 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc8dbfc-b2c1-45bf-8ae9-9ea1b04bfedc None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:56:17.750891 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 513/1042] 10.222.0.46 () {56 vars in 1109 bytes} [Thu May 26 14:56:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:56:17.756989 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af11a4a2-5954-4bc6-acf5-63ead6d22eb8 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:56:17.815351 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:56:17.820208 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981fb320-68a0-4d0d-880a-d35d1158ecda None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:17.821047 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981fb320-68a0-4d0d-880a-d35d1158ecda None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:17.821338 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981fb320-68a0-4d0d-880a-d35d1158ecda None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:17.822794 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 514/1043] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 14:56:17 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 14:56:17.828882 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fc0277-bc52-4572-9338-e0314f9b4357 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:17.829209 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fc0277-bc52-4572-9338-e0314f9b4357 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:17.829565 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fc0277-bc52-4572-9338-e0314f9b4357 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:17.847972 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-54fc0277-bc52-4572-9338-e0314f9b4357 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:56:17.849170 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-54fc0277-bc52-4572-9338-e0314f9b4357 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:56:17.884772 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af11a4a2-5954-4bc6-acf5-63ead6d22eb8 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:56:17.924850 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577fe901-0628-416a-915b-858ea01cd954 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:56:17.925686 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577fe901-0628-416a-915b-858ea01cd954 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:17.925974 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577fe901-0628-416a-915b-858ea01cd954 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:17.926254 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-577fe901-0628-416a-915b-858ea01cd954 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:17.930962 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 515/1044] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 14:56:17 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 14:56:17.939339 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577fe901-0628-416a-915b-858ea01cd954 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:56:17.940881 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-577fe901-0628-416a-915b-858ea01cd954 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:56:17.955112 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fc0277-bc52-4572-9338-e0314f9b4357 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:56:17.956472 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-577fe901-0628-416a-915b-858ea01cd954 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 14:56:17.956472 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:56:17.956472 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:56:17.956472 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:56:17.956472 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:56:17.956472 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:56:17.956472 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:56:17.956472 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:56:17.956472 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:56:17.956472 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:56:17.956472 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:56:17.956472 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:56:17.956472 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 14:56:17.956472 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 14:56:17.956472 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 14:56:17.957864 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 14:56:17.957864 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:56:17.957864 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:56:17.957864 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:56:17.957864 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:56:17.957864 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:56:17.957864 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:56:17.957864 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:56:17.957864 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 26 14:56:17.957864 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:56:17.957864 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 26 14:56:17.957864 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:56:17.957864 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:56:17.957864 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:56:17.957864 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 26 14:56:17.957864 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:56:17.957864 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 26 14:56:17.958826 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 14:56:17.958826 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 14:56:17.958826 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 14:56:17.958826 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 14:56:17.958826 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 14:56:17.958826 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 14:56:17.958826 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 14:56:17.958826 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 26 14:56:17.963467 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 530/1045] 10.222.0.46 () {56 vars in 1111 bytes} [Thu May 26 14:56:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:56:17.964638 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577fe901-0628-416a-915b-858ea01cd954 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:56:18.061205 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:56:18.068187 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fc0277-bc52-4572-9338-e0314f9b4357 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:56:18.079706 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-577fe901-0628-416a-915b-858ea01cd954 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:56:18.094405 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:56:18.112419 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b5d03f-0be5-46b6-bf62-7db2e069adcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:56:18.113284 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b5d03f-0be5-46b6-bf62-7db2e069adcd None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:18.113606 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b5d03f-0be5-46b6-bf62-7db2e069adcd None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:18.113891 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b5d03f-0be5-46b6-bf62-7db2e069adcd None admin] PATH_INFO: `/v3/services/image` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:18.114613 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b5d03f-0be5-46b6-bf62-7db2e069adcd None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:56:18.116243 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b5d03f-0be5-46b6-bf62-7db2e069adcd None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:56:18.121817 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9f4c26-b375-4a99-b74d-e75ccfb13386 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:56:18.122486 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9f4c26-b375-4a99-b74d-e75ccfb13386 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:18.122774 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9f4c26-b375-4a99-b74d-e75ccfb13386 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:18.123053 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9f4c26-b375-4a99-b74d-e75ccfb13386 None admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:18.123844 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9f4c26-b375-4a99-b74d-e75ccfb13386 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:56:18.124157 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9f4c26-b375-4a99-b74d-e75ccfb13386 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:56:18.125626 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9f4c26-b375-4a99-b74d-e75ccfb13386 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:56:18.130877 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-55b5d03f-0be5-46b6-bf62-7db2e069adcd None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 26 14:56:18.130877 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:56:18.130877 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 14:56:18.130877 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 14:56:18.130877 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 14:56:18.130877 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 14:56:18.130877 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 14:56:18.130877 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 14:56:18.130877 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 26 14:56:18.130877 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 26 14:56:18.130877 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 14:56:18.130877 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 26 14:56:18.130877 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:56:18.130877 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:56:18.130877 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:56:18.130877 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:56:18.130877 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:56:18.131891 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:56:18.131891 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:56:18.131891 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:56:18.131891 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:56:18.131891 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:56:18.131891 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:56:18.131891 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 14:56:18.131891 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 14:56:18.131891 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 14:56:18.131891 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 14:56:18.131891 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:56:18.131891 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:56:18.131891 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:56:18.131891 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:56:18.131891 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:56:18.131891 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:56:18.132728 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:56:18.132728 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 26 14:56:18.132728 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:56:18.132728 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 26 14:56:18.132728 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:56:18.132728 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:56:18.132728 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:56:18.132728 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 26 14:56:18.132728 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:56:18.132728 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 26 14:56:18.132728 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 14:56:18.132728 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 14:56:18.132728 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 26 14:56:18.132728 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 26 14:56:18.132728 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 516/1046] 10.222.0.46 () {56 vars in 1105 bytes} [Thu May 26 14:56:17 2022] GET /identity/v3/services/image => generated 86 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 14:56:18.146759 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 531/1047] 10.222.0.46 () {56 vars in 1148 bytes} [Thu May 26 14:56:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:56:18.147045 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b5d03f-0be5-46b6-bf62-7db2e069adcd None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:56:18.147409 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9f4c26-b375-4a99-b74d-e75ccfb13386 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:56:18.262360 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b5d03f-0be5-46b6-bf62-7db2e069adcd None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:56:18.264835 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9f4c26-b375-4a99-b74d-e75ccfb13386 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:56:18.301205 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711f5b4d-c0c5-403e-9c09-4d7cf89daaab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:56:18.301923 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711f5b4d-c0c5-403e-9c09-4d7cf89daaab None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:18.302206 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711f5b4d-c0c5-403e-9c09-4d7cf89daaab None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:18.302478 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711f5b4d-c0c5-403e-9c09-4d7cf89daaab None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:18.302710 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e8e955-b42d-402d-bf8d-3d939cc28818 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:56:18.303305 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711f5b4d-c0c5-403e-9c09-4d7cf89daaab None admin] RBAC: Adding query filter params (role.id=3593816f89a648809d5d7c9160fb87dc, scope.project.id=a17fc8227d4c4da6b2847b350e912ee3, user.id=d6639353c70449d1b3d10cd6487c3b00) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:56:18.303614 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711f5b4d-c0c5-403e-9c09-4d7cf89daaab None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:56:18.303944 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e8e955-b42d-402d-bf8d-3d939cc28818 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:18.304417 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e8e955-b42d-402d-bf8d-3d939cc28818 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:18.305406 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e8e955-b42d-402d-bf8d-3d939cc28818 None admin] PATH_INFO: `/v3/services` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:18.305782 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711f5b4d-c0c5-403e-9c09-4d7cf89daaab None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:56:18.306513 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e8e955-b42d-402d-bf8d-3d939cc28818 None admin] RBAC: Adding query filter params (name=image) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:56:18.307002 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e8e955-b42d-402d-bf8d-3d939cc28818 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:56:18.308819 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e8e955-b42d-402d-bf8d-3d939cc28818 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:56:18.312189 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 532/1048] 10.222.0.46 () {56 vars in 1374 bytes} [Thu May 26 14:56:18 2022] GET /identity/v3/role_assignments?user.id=d6639353c70449d1b3d10cd6487c3b00&scope.project.id=a17fc8227d4c4da6b2847b350e912ee3&role.id=3593816f89a648809d5d7c9160fb87dc => generated 259 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 14:56:18.316141 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 517/1049] 10.222.0.46 () {56 vars in 1108 bytes} [Thu May 26 14:56:18 2022] GET /identity/v3/services?name=image => generated 122 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:56:18.326946 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711f5b4d-c0c5-403e-9c09-4d7cf89daaab None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:56:18.369111 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:56:18.446355 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711f5b4d-c0c5-403e-9c09-4d7cf89daaab None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:56:18.490074 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f16a6f-3241-4137-bf48-de883eb6fd93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:56:18.490750 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f16a6f-3241-4137-bf48-de883eb6fd93 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:18.491030 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f16a6f-3241-4137-bf48-de883eb6fd93 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:18.491302 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f16a6f-3241-4137-bf48-de883eb6fd93 None admin] PATH_INFO: `/v3/services` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:18.492009 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f16a6f-3241-4137-bf48-de883eb6fd93 None admin] RBAC: Adding query filter params (type=image) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:56:18.494713 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f16a6f-3241-4137-bf48-de883eb6fd93 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:56:18.496124 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f16a6f-3241-4137-bf48-de883eb6fd93 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:56:18.505260 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 533/1050] 10.222.0.46 () {56 vars in 1108 bytes} [Thu May 26 14:56:18 2022] GET /identity/v3/services?type=image => generated 350 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:56:18.511978 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e8e955-b42d-402d-bf8d-3d939cc28818 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:56:18.543757 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:56:18.636732 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e8e955-b42d-402d-bf8d-3d939cc28818 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:56:18.674704 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af19a73-c7ac-4ab5-a129-8344380b92ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:56:18.675375 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af19a73-c7ac-4ab5-a129-8344380b92ae None admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:18.675658 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af19a73-c7ac-4ab5-a129-8344380b92ae None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:18.675934 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af19a73-c7ac-4ab5-a129-8344380b92ae None admin] PATH_INFO: `/v3/endpoints` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:18.676694 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af19a73-c7ac-4ab5-a129-8344380b92ae None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:56:18.678108 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af19a73-c7ac-4ab5-a129-8344380b92ae None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:56:18.705525 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 518/1051] 10.222.0.46 () {60 vars in 1144 bytes} [Thu May 26 14:56:18 2022] POST /identity/v3/endpoints => generated 327 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 14:56:19.737850 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb79ea7-169c-4fb0-8732-6db40487ed27 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:19.738409 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb79ea7-169c-4fb0-8732-6db40487ed27 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:19.738409 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb79ea7-169c-4fb0-8732-6db40487ed27 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:19.740033 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 534/1052] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 14:56:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 14:56:19.745427 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e47885-466d-4b7b-909c-1e3cafd7a8be None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:19.745938 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e47885-466d-4b7b-909c-1e3cafd7a8be None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:19.746031 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e47885-466d-4b7b-909c-1e3cafd7a8be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:19.763429 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-07e47885-466d-4b7b-909c-1e3cafd7a8be None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:56:19.764620 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-07e47885-466d-4b7b-909c-1e3cafd7a8be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:56:19.846391 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 519/1053] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 14:56:19 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 14:56:19.875611 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb79ea7-169c-4fb0-8732-6db40487ed27 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:56:19.984847 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:56:19.993599 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb79ea7-169c-4fb0-8732-6db40487ed27 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:56:20.034363 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f250221c-f771-4756-9fd9-6d61062f282a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:56:20.035172 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f250221c-f771-4756-9fd9-6d61062f282a None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:20.035454 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f250221c-f771-4756-9fd9-6d61062f282a None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:20.035734 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f250221c-f771-4756-9fd9-6d61062f282a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:20.048538 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f250221c-f771-4756-9fd9-6d61062f282a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 14:56:20.056380 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-f250221c-f771-4756-9fd9-6d61062f282a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 14:56:20.056380 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:56:20.056380 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:56:20.056380 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:56:20.056380 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:56:20.056380 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:56:20.056380 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:56:20.056380 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:56:20.056380 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:56:20.056380 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:56:20.056380 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:56:20.056380 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:56:20.056380 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 14:56:20.056380 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 14:56:20.056380 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 14:56:20.056380 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 14:56:20.057413 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 14:56:20.057413 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 14:56:20.057413 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 14:56:20.057413 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 14:56:20.057413 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:56:20.057413 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:56:20.057413 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:56:20.057413 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:56:20.057413 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:56:20.057413 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:56:20.057413 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:56:20.057413 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 26 14:56:20.057413 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:56:20.057413 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 26 14:56:20.057413 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:56:20.057413 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:56:20.058289 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:56:20.058289 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 26 14:56:20.058289 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:56:20.058289 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 26 14:56:20.058289 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 14:56:20.058289 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 14:56:20.058289 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 14:56:20.058289 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 14:56:20.058289 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 14:56:20.058289 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 14:56:20.058289 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 14:56:20.058289 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 26 14:56:20.061538 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 535/1054] 10.222.0.46 () {56 vars in 1096 bytes} [Thu May 26 14:56:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:56:20.063775 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e47885-466d-4b7b-909c-1e3cafd7a8be None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:56:20.073601 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:56:20.177722 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e47885-466d-4b7b-909c-1e3cafd7a8be None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:56:20.218203 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c60234-160c-46dc-ac34-768130d15eb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:56:20.218893 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c60234-160c-46dc-ac34-768130d15eb1 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:20.219182 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c60234-160c-46dc-ac34-768130d15eb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:20.219457 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c60234-160c-46dc-ac34-768130d15eb1 None admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:20.220302 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c60234-160c-46dc-ac34-768130d15eb1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:56:20.220627 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c60234-160c-46dc-ac34-768130d15eb1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:56:20.222128 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c60234-160c-46dc-ac34-768130d15eb1 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:56:20.231250 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 520/1055] 10.222.0.46 () {56 vars in 1099 bytes} [Thu May 26 14:56:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 14:56:20.236810 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f250221c-f771-4756-9fd9-6d61062f282a None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:56:20.345753 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f250221c-f771-4756-9fd9-6d61062f282a None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:56:20.391033 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cd51d1-2791-43ef-b583-17f94b1f8614 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:56:20.391933 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cd51d1-2791-43ef-b583-17f94b1f8614 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:20.392210 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cd51d1-2791-43ef-b583-17f94b1f8614 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:20.392481 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cd51d1-2791-43ef-b583-17f94b1f8614 None admin] PATH_INFO: `/v3/users/af259dcfa94146b5834c532e1496cfcf` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:20.412771 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3cd51d1-2791-43ef-b583-17f94b1f8614 None admin] RBAC: Authorizing `identity:get_user(user_id=af259dcfa94146b5834c532e1496cfcf)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:56:20.414288 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3cd51d1-2791-43ef-b583-17f94b1f8614 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:56:20.416217 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 536/1056] 10.222.0.46 () {56 vars in 1177 bytes} [Thu May 26 14:56:20 2022] GET /identity/v3/users/af259dcfa94146b5834c532e1496cfcf => generated 285 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:56:20.422213 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c60234-160c-46dc-ac34-768130d15eb1 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:56:20.498035 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:56:20.535222 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c60234-160c-46dc-ac34-768130d15eb1 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:56:20.560527 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039f97e9-411d-4e12-9538-ae7e722a6be6 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:20.560918 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039f97e9-411d-4e12-9538-ae7e722a6be6 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:20.561443 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039f97e9-411d-4e12-9538-ae7e722a6be6 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:20.564034 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 537/1057] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 14:56:20 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 14:56:20.571067 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de9a8c5-a81f-472f-956a-5a9492aad274 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:20.571403 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de9a8c5-a81f-472f-956a-5a9492aad274 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:20.571717 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de9a8c5-a81f-472f-956a-5a9492aad274 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:20.581763 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4dd22a-8f7a-420d-9a10-93ec5fe16e02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:56:20.582437 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4dd22a-8f7a-420d-9a10-93ec5fe16e02 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:20.582718 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4dd22a-8f7a-420d-9a10-93ec5fe16e02 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:20.582989 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4dd22a-8f7a-420d-9a10-93ec5fe16e02 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:20.592106 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-2de9a8c5-a81f-472f-956a-5a9492aad274 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:56:20.593367 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-2de9a8c5-a81f-472f-956a-5a9492aad274 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:56:20.607506 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:56:20.668386 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4dd22a-8f7a-420d-9a10-93ec5fe16e02 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:56:20.669887 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4dd22a-8f7a-420d-9a10-93ec5fe16e02 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:56:20.681487 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 538/1058] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 14:56:20 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 14:56:20.702822 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de9a8c5-a81f-472f-956a-5a9492aad274 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:56:20.754238 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:56:20.793472 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 521/1059] 10.222.0.46 () {58 vars in 1282 bytes} [Thu May 26 14:56:20 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 26 14:56:20.799406 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4dd22a-8f7a-420d-9a10-93ec5fe16e02 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:56:20.838436 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de9a8c5-a81f-472f-956a-5a9492aad274 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:56:20.881166 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd770e2-886d-4fa1-8aa0-06249936f1d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:56:20.882050 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd770e2-886d-4fa1-8aa0-06249936f1d2 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:20.882335 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd770e2-886d-4fa1-8aa0-06249936f1d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:20.882613 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd770e2-886d-4fa1-8aa0-06249936f1d2 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:20.895376 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd770e2-886d-4fa1-8aa0-06249936f1d2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 14:56:20.904709 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-8dd770e2-886d-4fa1-8aa0-06249936f1d2 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 26 14:56:20.904709 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:56:20.904709 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:56:20.904709 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:56:20.904709 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:56:20.904709 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:56:20.904709 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:56:20.904709 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:56:20.904709 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:56:20.904709 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:56:20.904709 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:56:20.904709 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:56:20.904709 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 14:56:20.904709 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 14:56:20.904709 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 14:56:20.904709 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 14:56:20.905898 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 14:56:20.905898 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 14:56:20.905898 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 14:56:20.905898 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 14:56:20.905898 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:56:20.905898 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:56:20.905898 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:56:20.905898 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:56:20.905898 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:56:20.905898 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:56:20.905898 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:56:20.905898 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 26 14:56:20.905898 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:56:20.905898 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 26 14:56:20.905898 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:56:20.905898 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:56:20.908022 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:56:20.908022 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 26 14:56:20.908022 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:56:20.908022 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 26 14:56:20.908022 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 14:56:20.908022 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 14:56:20.908022 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 14:56:20.908022 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 14:56:20.908022 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 14:56:20.908022 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 14:56:20.908022 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 26 14:56:20.908022 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 26 14:56:20.908022 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 539/1060] 10.222.0.46 () {56 vars in 1120 bytes} [Thu May 26 14:56:20 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:56:20.913225 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd770e2-886d-4fa1-8aa0-06249936f1d2 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:56:20.925669 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:56:20.932016 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4dd22a-8f7a-420d-9a10-93ec5fe16e02 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:56:20.972035 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c34f3e-6f88-4e46-97a0-e4667d8d9dc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:56:20.973020 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c34f3e-6f88-4e46-97a0-e4667d8d9dc1 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:20.973306 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c34f3e-6f88-4e46-97a0-e4667d8d9dc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:20.973617 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c34f3e-6f88-4e46-97a0-e4667d8d9dc1 None admin] PATH_INFO: `/v3/projects/5bcc4bc1b6ef4339b66e4943d0597e03` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:20.974860 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c34f3e-6f88-4e46-97a0-e4667d8d9dc1 None admin] RBAC: Authorizing `identity:get_project(project_id=5bcc4bc1b6ef4339b66e4943d0597e03)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:56:20.976317 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c34f3e-6f88-4e46-97a0-e4667d8d9dc1 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:56:20.978138 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 522/1061] 10.222.0.46 () {56 vars in 1186 bytes} [Thu May 26 14:56:20 2022] GET /identity/v3/projects/5bcc4bc1b6ef4339b66e4943d0597e03 => generated 311 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 14:56:20.984418 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c34f3e-6f88-4e46-97a0-e4667d8d9dc1 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:56:21.025910 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:56:21.041309 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd770e2-886d-4fa1-8aa0-06249936f1d2 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:56:21.076829 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67458edf-3d3f-43e2-9196-5504964579ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:56:21.077634 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67458edf-3d3f-43e2-9196-5504964579ca None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:21.077925 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67458edf-3d3f-43e2-9196-5504964579ca None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:21.078199 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67458edf-3d3f-43e2-9196-5504964579ca None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:21.078944 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67458edf-3d3f-43e2-9196-5504964579ca None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:56:21.079251 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67458edf-3d3f-43e2-9196-5504964579ca None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:56:21.080621 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67458edf-3d3f-43e2-9196-5504964579ca None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:56:21.093282 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 540/1062] 10.222.0.46 () {56 vars in 1115 bytes} [Thu May 26 14:56:20 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:56:21.093282 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8bd6b3-32c8-40d0-a2ae-08ef2f90d107 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:21.093282 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8bd6b3-32c8-40d0-a2ae-08ef2f90d107 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:21.093282 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8bd6b3-32c8-40d0-a2ae-08ef2f90d107 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:21.093282 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 541/1063] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 14:56:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 14:56:21.095517 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8bd6b3-32c8-40d0-a2ae-08ef2f90d107 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:56:21.098488 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c34f3e-6f88-4e46-97a0-e4667d8d9dc1 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:56:21.132619 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1f3237-4567-4065-a551-cb93802d1062 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:56:21.133881 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1f3237-4567-4065-a551-cb93802d1062 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:21.134248 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1f3237-4567-4065-a551-cb93802d1062 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:21.134534 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1f3237-4567-4065-a551-cb93802d1062 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:21.135364 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1f3237-4567-4065-a551-cb93802d1062 None admin] RBAC: Adding query filter params (role.id=a2e551773cbd4df084020676865b8222, scope.project.id=5bcc4bc1b6ef4339b66e4943d0597e03, user.id=af259dcfa94146b5834c532e1496cfcf) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:56:21.135680 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1f3237-4567-4065-a551-cb93802d1062 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:56:21.137133 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1f3237-4567-4065-a551-cb93802d1062 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:56:21.143546 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 523/1064] 10.222.0.46 () {56 vars in 1374 bytes} [Thu May 26 14:56:20 2022] GET /identity/v3/role_assignments?user.id=af259dcfa94146b5834c532e1496cfcf&scope.project.id=5bcc4bc1b6ef4339b66e4943d0597e03&role.id=a2e551773cbd4df084020676865b8222 => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 14:56:21.144893 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3aed17-6633-4e6d-afaa-53d0dded5990 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:21.145605 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3aed17-6633-4e6d-afaa-53d0dded5990 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:21.145826 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3aed17-6633-4e6d-afaa-53d0dded5990 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:21.164883 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-cd3aed17-6633-4e6d-afaa-53d0dded5990 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:56:21.166179 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-cd3aed17-6633-4e6d-afaa-53d0dded5990 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:56:21.177226 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:56:21.224290 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8bd6b3-32c8-40d0-a2ae-08ef2f90d107 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:56:21.253584 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 524/1065] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 14:56:21 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 14:56:21.274948 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588d0a37-9661-40a5-8ccd-15a0b0c892a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:56:21.276049 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588d0a37-9661-40a5-8ccd-15a0b0c892a2 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:21.276264 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588d0a37-9661-40a5-8ccd-15a0b0c892a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:21.276473 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588d0a37-9661-40a5-8ccd-15a0b0c892a2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:21.277018 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3aed17-6633-4e6d-afaa-53d0dded5990 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:56:21.291070 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588d0a37-9661-40a5-8ccd-15a0b0c892a2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:56:21.292648 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588d0a37-9661-40a5-8ccd-15a0b0c892a2 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:56:21.312130 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-588d0a37-9661-40a5-8ccd-15a0b0c892a2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 14:56:21.312130 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:56:21.312130 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:56:21.312130 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:56:21.312130 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:56:21.312130 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:56:21.312130 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:56:21.312130 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:56:21.312130 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:56:21.312130 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:56:21.312130 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:56:21.312130 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:56:21.312130 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 14:56:21.312130 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 14:56:21.312130 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 14:56:21.313716 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 14:56:21.313716 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:56:21.313716 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:56:21.313716 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:56:21.313716 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:56:21.313716 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:56:21.313716 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:56:21.313716 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:56:21.313716 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 26 14:56:21.313716 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:56:21.313716 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 26 14:56:21.313716 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:56:21.313716 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:56:21.313716 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:56:21.313716 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 26 14:56:21.313716 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:56:21.313716 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 26 14:56:21.314640 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 14:56:21.314640 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 14:56:21.314640 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 14:56:21.314640 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 26 14:56:21.314640 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 542/1066] 10.222.0.46 () {56 vars in 1108 bytes} [Thu May 26 14:56:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:56:21.315824 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:56:21.320187 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588d0a37-9661-40a5-8ccd-15a0b0c892a2 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:56:21.404149 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3aed17-6633-4e6d-afaa-53d0dded5990 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:56:21.441934 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38be6a4-f692-4084-81c7-6de9ac0ac4b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:56:21.442594 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38be6a4-f692-4084-81c7-6de9ac0ac4b3 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:21.442870 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38be6a4-f692-4084-81c7-6de9ac0ac4b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:21.443159 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38be6a4-f692-4084-81c7-6de9ac0ac4b3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:21.453089 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588d0a37-9661-40a5-8ccd-15a0b0c892a2 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:56:21.483731 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:56:21.500536 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324988cc-af12-42f4-b787-12160c6edd4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:56:21.501220 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324988cc-af12-42f4-b787-12160c6edd4c None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:21.501527 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324988cc-af12-42f4-b787-12160c6edd4c None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:21.501803 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-324988cc-af12-42f4-b787-12160c6edd4c None admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:21.502535 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324988cc-af12-42f4-b787-12160c6edd4c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:56:21.502837 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324988cc-af12-42f4-b787-12160c6edd4c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:56:21.504484 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-324988cc-af12-42f4-b787-12160c6edd4c None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:56:21.517738 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 543/1067] 10.222.0.46 () {56 vars in 1109 bytes} [Thu May 26 14:56:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 14:56:21.523794 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324988cc-af12-42f4-b787-12160c6edd4c None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:56:21.526876 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38be6a4-f692-4084-81c7-6de9ac0ac4b3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:56:21.528314 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38be6a4-f692-4084-81c7-6de9ac0ac4b3 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:56:21.558566 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:56:21.642889 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-324988cc-af12-42f4-b787-12160c6edd4c None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:56:21.643612 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 525/1068] 10.222.0.46 () {58 vars in 1282 bytes} [Thu May 26 14:56:21 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 26 14:56:21.647579 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38be6a4-f692-4084-81c7-6de9ac0ac4b3 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:56:21.683384 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c59518-a12a-4f83-a23f-e7802cebbe33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:56:21.684185 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c59518-a12a-4f83-a23f-e7802cebbe33 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:21.684472 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c59518-a12a-4f83-a23f-e7802cebbe33 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:21.685516 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c59518-a12a-4f83-a23f-e7802cebbe33 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:21.692716 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c59518-a12a-4f83-a23f-e7802cebbe33 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:56:21.694316 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c59518-a12a-4f83-a23f-e7802cebbe33 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:56:21.710310 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-09c59518-a12a-4f83-a23f-e7802cebbe33 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 26 14:56:21.710310 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:56:21.710310 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:56:21.710310 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:56:21.710310 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:56:21.710310 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:56:21.710310 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:56:21.710310 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:56:21.710310 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:56:21.710310 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:56:21.710310 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:56:21.710310 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:56:21.710310 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 14:56:21.710310 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 14:56:21.710310 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 14:56:21.710310 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 14:56:21.711939 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:56:21.711939 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:56:21.711939 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 14:56:21.711939 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 14:56:21.711939 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 14:56:21.711939 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 14:56:21.711939 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:56:21.711939 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:56:21.711939 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:56:21.711939 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:56:21.711939 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:56:21.711939 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 26 14:56:21.711939 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:56:21.711939 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 26 14:56:21.711939 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:56:21.711939 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:56:21.711939 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:56:21.711939 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 26 14:56:21.713546 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:56:21.713546 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 26 14:56:21.713546 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 14:56:21.713546 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 14:56:21.713546 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 14:56:21.713546 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 14:56:21.713546 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 14:56:21.713546 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 14:56:21.713546 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 26 14:56:21.713546 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 26 14:56:21.713546 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 544/1069] 10.222.0.46 () {56 vars in 1093 bytes} [Thu May 26 14:56:21 2022] GET /identity/v3/users/nova => generated 82 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 14:56:21.717998 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c59518-a12a-4f83-a23f-e7802cebbe33 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:56:21.758795 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38be6a4-f692-4084-81c7-6de9ac0ac4b3 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:56:21.768300 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:56:21.792316 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:56:21.801147 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258266d4-3402-42eb-bd62-bd3b79eb1431 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:56:21.802091 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258266d4-3402-42eb-bd62-bd3b79eb1431 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:21.802377 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258266d4-3402-42eb-bd62-bd3b79eb1431 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:21.802656 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258266d4-3402-42eb-bd62-bd3b79eb1431 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:21.814963 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258266d4-3402-42eb-bd62-bd3b79eb1431 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:56:21.816539 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258266d4-3402-42eb-bd62-bd3b79eb1431 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:56:21.829372 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c59518-a12a-4f83-a23f-e7802cebbe33 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:56:21.837002 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-258266d4-3402-42eb-bd62-bd3b79eb1431 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 14:56:21.837002 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:56:21.837002 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:56:21.837002 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:56:21.837002 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:56:21.837002 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:56:21.837002 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:56:21.837002 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:56:21.837002 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:56:21.837002 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:56:21.837002 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:56:21.837002 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:56:21.837002 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 14:56:21.837002 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 14:56:21.837002 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 14:56:21.838258 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 14:56:21.838258 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:56:21.838258 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:56:21.838258 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:56:21.838258 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:56:21.838258 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:56:21.838258 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:56:21.838258 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:56:21.838258 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 26 14:56:21.838258 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:56:21.838258 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 26 14:56:21.838258 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:56:21.838258 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:56:21.838258 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:56:21.838258 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 26 14:56:21.838258 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:56:21.838258 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 26 14:56:21.839179 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 14:56:21.839179 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 14:56:21.839179 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 14:56:21.839179 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 26 14:56:21.839179 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 526/1070] 10.222.0.46 () {56 vars in 1108 bytes} [Thu May 26 14:56:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 14:56:21.844162 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258266d4-3402-42eb-bd62-bd3b79eb1431 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:56:21.870864 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047dedd4-35e9-4f77-96ba-0e5d0f38ca7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:56:21.871539 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047dedd4-35e9-4f77-96ba-0e5d0f38ca7c None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:21.871838 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047dedd4-35e9-4f77-96ba-0e5d0f38ca7c None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:21.872109 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-047dedd4-35e9-4f77-96ba-0e5d0f38ca7c None admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:21.872956 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047dedd4-35e9-4f77-96ba-0e5d0f38ca7c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:56:21.873269 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047dedd4-35e9-4f77-96ba-0e5d0f38ca7c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:56:21.874731 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-047dedd4-35e9-4f77-96ba-0e5d0f38ca7c None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:56:21.894240 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 545/1071] 10.222.0.46 () {56 vars in 1133 bytes} [Thu May 26 14:56:21 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 14:56:21.900221 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047dedd4-35e9-4f77-96ba-0e5d0f38ca7c None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:56:21.953727 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258266d4-3402-42eb-bd62-bd3b79eb1431 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:56:21.994076 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b9e14f-77bf-4aa2-96a9-ee060567f431 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:56:21.995133 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b9e14f-77bf-4aa2-96a9-ee060567f431 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:21.995422 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b9e14f-77bf-4aa2-96a9-ee060567f431 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:21.996001 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b9e14f-77bf-4aa2-96a9-ee060567f431 None admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:21.996752 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b9e14f-77bf-4aa2-96a9-ee060567f431 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:56:21.997065 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b9e14f-77bf-4aa2-96a9-ee060567f431 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:56:21.998503 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b9e14f-77bf-4aa2-96a9-ee060567f431 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:56:22.018253 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:56:22.021920 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-047dedd4-35e9-4f77-96ba-0e5d0f38ca7c None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:56:22.022310 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 527/1072] 10.222.0.46 () {56 vars in 1109 bytes} [Thu May 26 14:56:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 14:56:22.063325 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b568a516-13f4-4863-9b95-9b58bf030b73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:56:22.064053 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b568a516-13f4-4863-9b95-9b58bf030b73 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:22.064332 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b568a516-13f4-4863-9b95-9b58bf030b73 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:22.064605 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b568a516-13f4-4863-9b95-9b58bf030b73 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:22.077320 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b568a516-13f4-4863-9b95-9b58bf030b73 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:56:22.078847 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b568a516-13f4-4863-9b95-9b58bf030b73 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:56:22.098633 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-b568a516-13f4-4863-9b95-9b58bf030b73 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 14:56:22.098633 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:56:22.098633 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:56:22.098633 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:56:22.098633 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:56:22.098633 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:56:22.098633 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:56:22.098633 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:56:22.098633 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:56:22.098633 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:56:22.098633 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:56:22.098633 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:56:22.098633 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 14:56:22.098633 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 14:56:22.098633 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 14:56:22.099804 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 14:56:22.099804 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:56:22.099804 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:56:22.099804 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:56:22.099804 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:56:22.099804 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:56:22.099804 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:56:22.099804 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:56:22.099804 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 26 14:56:22.099804 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:56:22.099804 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 26 14:56:22.099804 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:56:22.099804 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:56:22.099804 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:56:22.099804 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 26 14:56:22.099804 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:56:22.099804 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 26 14:56:22.100720 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 14:56:22.100720 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 14:56:22.100720 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 14:56:22.100720 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 26 14:56:22.100720 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 546/1073] 10.222.0.46 () {56 vars in 1108 bytes} [Thu May 26 14:56:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 14:56:22.106795 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b9e14f-77bf-4aa2-96a9-ee060567f431 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:56:22.161501 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:56:22.230851 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b9e14f-77bf-4aa2-96a9-ee060567f431 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:56:22.273539 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284251d3-f230-4bb4-95af-ea7a936268c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:56:22.274511 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284251d3-f230-4bb4-95af-ea7a936268c7 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:22.274801 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284251d3-f230-4bb4-95af-ea7a936268c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:22.275077 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284251d3-f230-4bb4-95af-ea7a936268c7 None admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:22.275822 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284251d3-f230-4bb4-95af-ea7a936268c7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:56:22.276129 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284251d3-f230-4bb4-95af-ea7a936268c7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:56:22.277636 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284251d3-f230-4bb4-95af-ea7a936268c7 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:56:22.292242 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 528/1074] 10.222.0.46 () {56 vars in 1109 bytes} [Thu May 26 14:56:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:56:22.297899 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b568a516-13f4-4863-9b95-9b58bf030b73 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:56:22.408549 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b568a516-13f4-4863-9b95-9b58bf030b73 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:56:22.444640 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b7a53e-33bd-4370-bab5-32d31b2f13ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:56:22.445528 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b7a53e-33bd-4370-bab5-32d31b2f13ff None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:22.445789 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b7a53e-33bd-4370-bab5-32d31b2f13ff None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:22.446067 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b7a53e-33bd-4370-bab5-32d31b2f13ff None admin] PATH_INFO: `/v3/projects/service` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:22.453277 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b7a53e-33bd-4370-bab5-32d31b2f13ff None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:56:22.454762 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b7a53e-33bd-4370-bab5-32d31b2f13ff None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:56:22.469643 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-c9b7a53e-33bd-4370-bab5-32d31b2f13ff None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 14:56:22.469643 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:56:22.469643 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:56:22.469643 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:56:22.469643 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:56:22.469643 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:56:22.469643 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:56:22.469643 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:56:22.469643 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:56:22.469643 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:56:22.469643 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:56:22.469643 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:56:22.469643 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 14:56:22.469643 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 14:56:22.469643 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 14:56:22.470669 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 14:56:22.470669 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:56:22.470669 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:56:22.470669 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:56:22.470669 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:56:22.470669 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:56:22.470669 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:56:22.470669 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:56:22.470669 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 26 14:56:22.470669 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:56:22.470669 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 26 14:56:22.470669 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:56:22.470669 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:56:22.470669 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:56:22.470669 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 26 14:56:22.470669 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:56:22.470669 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 26 14:56:22.471589 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 14:56:22.471589 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 14:56:22.471589 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 14:56:22.471589 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 14:56:22.471589 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 14:56:22.471589 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 14:56:22.471589 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 14:56:22.471589 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 26 14:56:22.471589 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 547/1075] 10.222.0.46 () {56 vars in 1111 bytes} [Thu May 26 14:56:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:56:22.477396 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284251d3-f230-4bb4-95af-ea7a936268c7 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:56:22.588670 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284251d3-f230-4bb4-95af-ea7a936268c7 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:56:22.590273 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:56:22.632113 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f8b323-5426-4dfa-ae01-75ffb6e40bcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:56:22.633133 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f8b323-5426-4dfa-ae01-75ffb6e40bcb None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:22.633257 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f8b323-5426-4dfa-ae01-75ffb6e40bcb None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:22.633475 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f8b323-5426-4dfa-ae01-75ffb6e40bcb None admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:22.634379 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f8b323-5426-4dfa-ae01-75ffb6e40bcb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:56:22.635724 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f8b323-5426-4dfa-ae01-75ffb6e40bcb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:56:22.636195 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f8b323-5426-4dfa-ae01-75ffb6e40bcb None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:56:22.648812 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 529/1076] 10.222.0.46 () {56 vars in 1148 bytes} [Thu May 26 14:56:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:56:22.654739 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b7a53e-33bd-4370-bab5-32d31b2f13ff None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:56:22.668808 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:56:22.758371 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b7a53e-33bd-4370-bab5-32d31b2f13ff None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:56:22.793878 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09006e62-f9f9-4f96-bb54-7f97786dbeea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:56:22.794773 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09006e62-f9f9-4f96-bb54-7f97786dbeea None admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:22.795054 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09006e62-f9f9-4f96-bb54-7f97786dbeea None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:22.795329 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09006e62-f9f9-4f96-bb54-7f97786dbeea None admin] PATH_INFO: `/v3/projects/a17fc8227d4c4da6b2847b350e912ee3/users/d6639353c70449d1b3d10cd6487c3b00/roles/3593816f89a648809d5d7c9160fb87dc` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:22.797718 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09006e62-f9f9-4f96-bb54-7f97786dbeea None admin] RBAC: Authorizing `identity:create_grant(project_id=a17fc8227d4c4da6b2847b350e912ee3, user_id=d6639353c70449d1b3d10cd6487c3b00, role_id=3593816f89a648809d5d7c9160fb87dc)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:56:22.799166 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09006e62-f9f9-4f96-bb54-7f97786dbeea None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:56:22.813222 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 548/1077] 10.222.0.46 () {58 vars in 1439 bytes} [Thu May 26 14:56:22 2022] PUT /identity/v3/projects/a17fc8227d4c4da6b2847b350e912ee3/users/d6639353c70449d1b3d10cd6487c3b00/roles/3593816f89a648809d5d7c9160fb87dc => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 14:56:23.376022 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6995fda6-8102-4f65-8b1d-94d4262a0113 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:23.376581 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6995fda6-8102-4f65-8b1d-94d4262a0113 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:23.376581 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6995fda6-8102-4f65-8b1d-94d4262a0113 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:23.377976 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 530/1078] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 14:56:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 14:56:23.384206 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edabe715-6d92-4198-a8ec-bbd620366d11 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:23.384625 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edabe715-6d92-4198-a8ec-bbd620366d11 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:23.384763 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edabe715-6d92-4198-a8ec-bbd620366d11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:23.402073 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-edabe715-6d92-4198-a8ec-bbd620366d11 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:56:23.403271 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-edabe715-6d92-4198-a8ec-bbd620366d11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:56:23.465543 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:56:23.471821 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 549/1079] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 14:56:23 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 14:56:23.493161 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6995fda6-8102-4f65-8b1d-94d4262a0113 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:56:23.593331 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6995fda6-8102-4f65-8b1d-94d4262a0113 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:56:23.631433 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cad487e-15d8-4d67-93fc-9f8bfd3cb773 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:56:23.632167 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cad487e-15d8-4d67-93fc-9f8bfd3cb773 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:23.632446 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cad487e-15d8-4d67-93fc-9f8bfd3cb773 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:23.632716 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cad487e-15d8-4d67-93fc-9f8bfd3cb773 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:23.642810 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cad487e-15d8-4d67-93fc-9f8bfd3cb773 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 14:56:23.650967 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-1cad487e-15d8-4d67-93fc-9f8bfd3cb773 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 14:56:23.650967 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:56:23.650967 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:56:23.650967 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:56:23.650967 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:56:23.650967 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:56:23.650967 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:56:23.650967 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:56:23.650967 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:56:23.650967 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:56:23.650967 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:56:23.650967 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:56:23.650967 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 14:56:23.650967 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 14:56:23.650967 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 14:56:23.650967 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 14:56:23.651908 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 14:56:23.651908 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 14:56:23.651908 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 14:56:23.651908 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 14:56:23.651908 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:56:23.651908 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:56:23.651908 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:56:23.651908 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:56:23.651908 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:56:23.651908 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:56:23.651908 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:56:23.651908 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 26 14:56:23.651908 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:56:23.651908 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 26 14:56:23.651908 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:56:23.651908 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:56:23.652964 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:56:23.652964 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 26 14:56:23.652964 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:56:23.652964 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 26 14:56:23.652964 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 14:56:23.652964 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 14:56:23.652964 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 14:56:23.652964 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 14:56:23.652964 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 14:56:23.652964 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 14:56:23.652964 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 14:56:23.652964 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 26 14:56:23.652964 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 531/1080] 10.222.0.46 () {56 vars in 1096 bytes} [Thu May 26 14:56:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:56:23.658098 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edabe715-6d92-4198-a8ec-bbd620366d11 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:56:23.753526 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edabe715-6d92-4198-a8ec-bbd620366d11 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:56:23.787666 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ddeff6-1592-4db9-8e1c-82719e1c3c0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:56:23.788331 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ddeff6-1592-4db9-8e1c-82719e1c3c0b None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:23.788614 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ddeff6-1592-4db9-8e1c-82719e1c3c0b None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:23.789470 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ddeff6-1592-4db9-8e1c-82719e1c3c0b None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:23.790227 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ddeff6-1592-4db9-8e1c-82719e1c3c0b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:56:23.790533 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ddeff6-1592-4db9-8e1c-82719e1c3c0b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:56:23.791902 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ddeff6-1592-4db9-8e1c-82719e1c3c0b None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:56:23.800006 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 550/1081] 10.222.0.46 () {56 vars in 1099 bytes} [Thu May 26 14:56:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:56:23.806551 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cad487e-15d8-4d67-93fc-9f8bfd3cb773 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:56:23.831270 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:56:23.905520 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cad487e-15d8-4d67-93fc-9f8bfd3cb773 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:56:23.941768 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc471fef-a5b3-47bc-8f92-b64c3728a791 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:56:23.942640 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc471fef-a5b3-47bc-8f92-b64c3728a791 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:23.942917 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc471fef-a5b3-47bc-8f92-b64c3728a791 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:23.943189 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc471fef-a5b3-47bc-8f92-b64c3728a791 None admin] PATH_INFO: `/v3/users/af259dcfa94146b5834c532e1496cfcf` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:23.944360 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc471fef-a5b3-47bc-8f92-b64c3728a791 None admin] RBAC: Authorizing `identity:get_user(user_id=af259dcfa94146b5834c532e1496cfcf)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:56:23.945850 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc471fef-a5b3-47bc-8f92-b64c3728a791 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:56:23.947646 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 532/1082] 10.222.0.46 () {56 vars in 1177 bytes} [Thu May 26 14:56:23 2022] GET /identity/v3/users/af259dcfa94146b5834c532e1496cfcf => generated 285 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:56:23.953270 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ddeff6-1592-4db9-8e1c-82719e1c3c0b None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:56:24.021608 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:56:24.061367 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ddeff6-1592-4db9-8e1c-82719e1c3c0b None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:56:24.093527 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f7dc38-bb81-4ab6-a43a-fe6e9e9a558d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:56:24.094382 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f7dc38-bb81-4ab6-a43a-fe6e9e9a558d None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:24.094661 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f7dc38-bb81-4ab6-a43a-fe6e9e9a558d None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:24.094931 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f7dc38-bb81-4ab6-a43a-fe6e9e9a558d None admin] PATH_INFO: `/v3/projects/5bcc4bc1b6ef4339b66e4943d0597e03` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:24.096088 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f7dc38-bb81-4ab6-a43a-fe6e9e9a558d None admin] RBAC: Authorizing `identity:get_project(project_id=5bcc4bc1b6ef4339b66e4943d0597e03)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:56:24.097539 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f7dc38-bb81-4ab6-a43a-fe6e9e9a558d None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:56:24.099233 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 551/1083] 10.222.0.46 () {56 vars in 1186 bytes} [Thu May 26 14:56:23 2022] GET /identity/v3/projects/5bcc4bc1b6ef4339b66e4943d0597e03 => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:56:24.105692 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc471fef-a5b3-47bc-8f92-b64c3728a791 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:56:24.194137 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:56:24.204033 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc471fef-a5b3-47bc-8f92-b64c3728a791 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:56:24.237958 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c69a10b-5711-4ca1-8a6e-1d48b270b8fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:56:24.238840 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c69a10b-5711-4ca1-8a6e-1d48b270b8fc None admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:24.239122 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c69a10b-5711-4ca1-8a6e-1d48b270b8fc None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:24.239394 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c69a10b-5711-4ca1-8a6e-1d48b270b8fc None admin] PATH_INFO: `/v3/projects/5bcc4bc1b6ef4339b66e4943d0597e03/users/af259dcfa94146b5834c532e1496cfcf/roles/a2e551773cbd4df084020676865b8222` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:24.241304 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c69a10b-5711-4ca1-8a6e-1d48b270b8fc None admin] RBAC: Authorizing `identity:create_grant(project_id=5bcc4bc1b6ef4339b66e4943d0597e03, user_id=af259dcfa94146b5834c532e1496cfcf, role_id=a2e551773cbd4df084020676865b8222)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:56:24.242899 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c69a10b-5711-4ca1-8a6e-1d48b270b8fc None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:56:24.261772 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 533/1084] 10.222.0.46 () {58 vars in 1439 bytes} [Thu May 26 14:56:24 2022] PUT /identity/v3/projects/5bcc4bc1b6ef4339b66e4943d0597e03/users/af259dcfa94146b5834c532e1496cfcf/roles/a2e551773cbd4df084020676865b8222 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 14:56:25.052969 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432daf6c-2ac0-4ada-9e7c-7384e860897d None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:25.053511 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432daf6c-2ac0-4ada-9e7c-7384e860897d None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:25.053634 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432daf6c-2ac0-4ada-9e7c-7384e860897d None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:25.055546 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 552/1085] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 14:56:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 14:56:25.060977 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6077d8-f9b3-47e5-bf83-75bcd9585b18 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:25.061407 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6077d8-f9b3-47e5-bf83-75bcd9585b18 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:25.061562 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6077d8-f9b3-47e5-bf83-75bcd9585b18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:25.078968 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-8b6077d8-f9b3-47e5-bf83-75bcd9585b18 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:56:25.080159 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-8b6077d8-f9b3-47e5-bf83-75bcd9585b18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:56:25.145284 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 534/1086] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 14:56:25 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 26 14:56:25.167722 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432daf6c-2ac0-4ada-9e7c-7384e860897d None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:56:25.271768 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432daf6c-2ac0-4ada-9e7c-7384e860897d None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:56:25.299698 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:56:25.306461 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e745e2-59de-41b4-a6ff-275b98460306 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:56:25.307215 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e745e2-59de-41b4-a6ff-275b98460306 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:25.307498 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e745e2-59de-41b4-a6ff-275b98460306 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:25.307775 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e745e2-59de-41b4-a6ff-275b98460306 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:25.318308 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e745e2-59de-41b4-a6ff-275b98460306 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 14:56:25.327000 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-62e745e2-59de-41b4-a6ff-275b98460306 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 26 14:56:25.327000 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:56:25.327000 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:56:25.327000 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:56:25.327000 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:56:25.327000 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:56:25.327000 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:56:25.327000 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:56:25.327000 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:56:25.327000 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:56:25.327000 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:56:25.327000 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:56:25.327000 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 14:56:25.327000 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 14:56:25.327000 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 14:56:25.327000 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 14:56:25.328157 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 14:56:25.328157 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 14:56:25.328157 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 14:56:25.328157 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 14:56:25.328157 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:56:25.328157 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:56:25.328157 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:56:25.328157 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:56:25.328157 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:56:25.328157 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:56:25.328157 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:56:25.328157 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 26 14:56:25.328157 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:56:25.328157 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 26 14:56:25.328157 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:56:25.328157 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:56:25.329117 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:56:25.329117 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 26 14:56:25.329117 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:56:25.329117 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 26 14:56:25.329117 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 14:56:25.329117 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 14:56:25.329117 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 14:56:25.329117 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 14:56:25.329117 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 14:56:25.329117 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 14:56:25.329117 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 26 14:56:25.329117 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 26 14:56:25.329117 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 553/1087] 10.222.0.46 () {56 vars in 1120 bytes} [Thu May 26 14:56:25 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:56:25.334123 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6077d8-f9b3-47e5-bf83-75bcd9585b18 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:56:25.415181 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:56:25.434982 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6077d8-f9b3-47e5-bf83-75bcd9585b18 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:56:25.467514 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37a49fc-4d9d-43bb-a1ed-20a6c86e3c7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:56:25.468212 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37a49fc-4d9d-43bb-a1ed-20a6c86e3c7d None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:25.468524 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37a49fc-4d9d-43bb-a1ed-20a6c86e3c7d None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:25.468794 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37a49fc-4d9d-43bb-a1ed-20a6c86e3c7d None admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:25.469558 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37a49fc-4d9d-43bb-a1ed-20a6c86e3c7d None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:56:25.469862 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37a49fc-4d9d-43bb-a1ed-20a6c86e3c7d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:56:25.471224 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37a49fc-4d9d-43bb-a1ed-20a6c86e3c7d None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:56:25.484089 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 535/1088] 10.222.0.46 () {56 vars in 1115 bytes} [Thu May 26 14:56:25 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 14:56:25.485336 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e745e2-59de-41b4-a6ff-275b98460306 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:56:25.581543 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e745e2-59de-41b4-a6ff-275b98460306 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:56:25.618549 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ba9b5b-2cdb-4c83-b8a9-427cd912c106 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:56:25.619374 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ba9b5b-2cdb-4c83-b8a9-427cd912c106 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:25.619725 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ba9b5b-2cdb-4c83-b8a9-427cd912c106 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:25.620009 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ba9b5b-2cdb-4c83-b8a9-427cd912c106 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:25.630628 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ba9b5b-2cdb-4c83-b8a9-427cd912c106 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:56:25.632074 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ba9b5b-2cdb-4c83-b8a9-427cd912c106 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:56:25.649358 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-29ba9b5b-2cdb-4c83-b8a9-427cd912c106 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 14:56:25.649358 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:56:25.649358 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:56:25.649358 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:56:25.649358 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:56:25.649358 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:56:25.649358 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:56:25.649358 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:56:25.649358 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:56:25.649358 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:56:25.649358 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:56:25.649358 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:56:25.649358 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 14:56:25.649358 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 14:56:25.649358 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 14:56:25.650284 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 14:56:25.650284 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:56:25.650284 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:56:25.650284 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:56:25.650284 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:56:25.650284 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:56:25.650284 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:56:25.650284 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:56:25.650284 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 26 14:56:25.650284 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:56:25.650284 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 26 14:56:25.650284 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:56:25.650284 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:56:25.650284 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:56:25.650284 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 26 14:56:25.650284 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:56:25.650284 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 26 14:56:25.651359 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 14:56:25.651359 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 14:56:25.651359 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 14:56:25.651359 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 26 14:56:25.651359 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 554/1089] 10.222.0.46 () {56 vars in 1108 bytes} [Thu May 26 14:56:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:56:25.656303 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37a49fc-4d9d-43bb-a1ed-20a6c86e3c7d None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:56:25.753688 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37a49fc-4d9d-43bb-a1ed-20a6c86e3c7d None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:56:25.786905 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65686e74-37fa-41a4-a0b2-c929171f907e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:56:25.787565 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65686e74-37fa-41a4-a0b2-c929171f907e None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:25.787846 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65686e74-37fa-41a4-a0b2-c929171f907e None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:25.788120 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65686e74-37fa-41a4-a0b2-c929171f907e None admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:25.789493 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65686e74-37fa-41a4-a0b2-c929171f907e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:56:25.789805 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65686e74-37fa-41a4-a0b2-c929171f907e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:56:25.791161 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65686e74-37fa-41a4-a0b2-c929171f907e None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:56:25.802658 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 536/1090] 10.222.0.46 () {56 vars in 1109 bytes} [Thu May 26 14:56:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:56:25.808240 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ba9b5b-2cdb-4c83-b8a9-427cd912c106 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:56:25.851317 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:56:25.908301 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ba9b5b-2cdb-4c83-b8a9-427cd912c106 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:56:25.942748 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27949f9-cd17-474a-b0f9-ae276b26d3dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:56:25.943561 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27949f9-cd17-474a-b0f9-ae276b26d3dd None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:25.943844 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27949f9-cd17-474a-b0f9-ae276b26d3dd None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:25.944121 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27949f9-cd17-474a-b0f9-ae276b26d3dd None admin] PATH_INFO: `/v3/users/nova` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:25.952234 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27949f9-cd17-474a-b0f9-ae276b26d3dd None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:56:25.953772 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27949f9-cd17-474a-b0f9-ae276b26d3dd None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:56:25.969722 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-f27949f9-cd17-474a-b0f9-ae276b26d3dd None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 26 14:56:25.969722 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:56:25.969722 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:56:25.969722 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:56:25.969722 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:56:25.969722 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:56:25.969722 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:56:25.969722 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:56:25.969722 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:56:25.969722 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:56:25.969722 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:56:25.969722 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:56:25.969722 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 14:56:25.969722 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 14:56:25.969722 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 14:56:25.969722 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 14:56:25.970923 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:56:25.970923 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:56:25.970923 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 14:56:25.970923 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 14:56:25.970923 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 14:56:25.970923 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 14:56:25.970923 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:56:25.970923 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:56:25.970923 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:56:25.970923 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:56:25.970923 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:56:25.970923 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 26 14:56:25.970923 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:56:25.970923 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 26 14:56:25.970923 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:56:25.970923 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:56:25.970923 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:56:25.970923 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 26 14:56:25.971911 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:56:25.971911 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 26 14:56:25.971911 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 14:56:25.971911 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 14:56:25.971911 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 14:56:25.971911 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 14:56:25.971911 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 14:56:25.971911 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 14:56:25.971911 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 26 14:56:25.971911 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 26 14:56:25.971911 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 555/1091] 10.222.0.46 () {56 vars in 1093 bytes} [Thu May 26 14:56:25 2022] GET /identity/v3/users/nova => generated 82 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:56:25.977710 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65686e74-37fa-41a4-a0b2-c929171f907e None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:56:25.980648 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:56:26.082038 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65686e74-37fa-41a4-a0b2-c929171f907e None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:56:26.117146 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc025b2-f433-42ec-ba6f-9cca4c2834e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:56:26.117781 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc025b2-f433-42ec-ba6f-9cca4c2834e2 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:26.118069 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc025b2-f433-42ec-ba6f-9cca4c2834e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:26.118349 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc025b2-f433-42ec-ba6f-9cca4c2834e2 None admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:26.119203 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc025b2-f433-42ec-ba6f-9cca4c2834e2 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:56:26.119518 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc025b2-f433-42ec-ba6f-9cca4c2834e2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:56:26.120974 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc025b2-f433-42ec-ba6f-9cca4c2834e2 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:56:26.140841 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 537/1092] 10.222.0.46 () {56 vars in 1133 bytes} [Thu May 26 14:56:25 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:56:26.146628 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27949f9-cd17-474a-b0f9-ae276b26d3dd None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:56:26.249531 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27949f9-cd17-474a-b0f9-ae276b26d3dd None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:56:26.251520 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:56:26.290302 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234ebbe8-b9a0-4b2b-9798-082999ac7d9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:56:26.290988 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234ebbe8-b9a0-4b2b-9798-082999ac7d9c None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:26.291274 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234ebbe8-b9a0-4b2b-9798-082999ac7d9c None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:26.291553 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234ebbe8-b9a0-4b2b-9798-082999ac7d9c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:26.365821 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234ebbe8-b9a0-4b2b-9798-082999ac7d9c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:56:26.367395 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234ebbe8-b9a0-4b2b-9798-082999ac7d9c None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:56:26.469769 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 556/1093] 10.222.0.46 () {58 vars in 1282 bytes} [Thu May 26 14:56:26 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 26 14:56:26.475506 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234ebbe8-b9a0-4b2b-9798-082999ac7d9c None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:56:26.476155 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acc50ce-c698-4a7a-94e4-d737fa3d77be None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:26.476512 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acc50ce-c698-4a7a-94e4-d737fa3d77be None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:26.476854 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acc50ce-c698-4a7a-94e4-d737fa3d77be None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:26.477604 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:56:26.478421 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 538/1094] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 14:56:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 14:56:26.483951 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485eb7f9-c750-4582-99de-5fb758591508 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:26.484241 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485eb7f9-c750-4582-99de-5fb758591508 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:26.484519 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485eb7f9-c750-4582-99de-5fb758591508 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:26.502470 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-485eb7f9-c750-4582-99de-5fb758591508 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:56:26.503652 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-485eb7f9-c750-4582-99de-5fb758591508 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:56:26.561873 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:56:26.574899 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 539/1095] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 14:56:26 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 26 14:56:26.577428 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234ebbe8-b9a0-4b2b-9798-082999ac7d9c None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:56:26.598437 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485eb7f9-c750-4582-99de-5fb758591508 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:56:26.611142 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30eec4c4-14ca-4153-a877-1c971fe5e621 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:56:26.611942 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30eec4c4-14ca-4153-a877-1c971fe5e621 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:26.612225 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30eec4c4-14ca-4153-a877-1c971fe5e621 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:26.612500 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30eec4c4-14ca-4153-a877-1c971fe5e621 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:26.623933 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30eec4c4-14ca-4153-a877-1c971fe5e621 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:56:26.625325 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30eec4c4-14ca-4153-a877-1c971fe5e621 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:56:26.643678 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-30eec4c4-14ca-4153-a877-1c971fe5e621 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 14:56:26.643678 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:56:26.643678 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:56:26.643678 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:56:26.643678 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:56:26.643678 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:56:26.643678 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:56:26.643678 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:56:26.643678 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:56:26.643678 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:56:26.643678 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:56:26.643678 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:56:26.643678 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 14:56:26.643678 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 14:56:26.643678 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 14:56:26.644713 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 14:56:26.644713 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:56:26.644713 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:56:26.644713 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:56:26.644713 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:56:26.644713 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:56:26.644713 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:56:26.644713 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:56:26.644713 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 26 14:56:26.644713 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:56:26.644713 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 26 14:56:26.644713 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:56:26.644713 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:56:26.644713 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:56:26.644713 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 26 14:56:26.644713 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:56:26.644713 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 26 14:56:26.645644 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 14:56:26.645644 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 14:56:26.645644 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 14:56:26.645644 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 26 14:56:26.645644 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 557/1096] 10.222.0.46 () {56 vars in 1108 bytes} [Thu May 26 14:56:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 14:56:26.650468 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30eec4c4-14ca-4153-a877-1c971fe5e621 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:56:26.705170 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485eb7f9-c750-4582-99de-5fb758591508 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:56:26.707583 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:56:26.741174 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd0d7a2-4da0-4d3b-8e7d-e3186f71a8d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:56:26.742512 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd0d7a2-4da0-4d3b-8e7d-e3186f71a8d2 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:26.742797 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd0d7a2-4da0-4d3b-8e7d-e3186f71a8d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:26.743069 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd0d7a2-4da0-4d3b-8e7d-e3186f71a8d2 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:26.751747 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30eec4c4-14ca-4153-a877-1c971fe5e621 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:56:26.753259 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd0d7a2-4da0-4d3b-8e7d-e3186f71a8d2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 14:56:26.761875 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-8bd0d7a2-4da0-4d3b-8e7d-e3186f71a8d2 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 14:56:26.761875 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:56:26.761875 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:56:26.761875 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:56:26.761875 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:56:26.761875 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:56:26.761875 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:56:26.761875 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:56:26.761875 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:56:26.761875 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:56:26.761875 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:56:26.761875 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:56:26.761875 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 14:56:26.761875 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 14:56:26.761875 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 14:56:26.761875 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 14:56:26.762842 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 14:56:26.762842 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 14:56:26.762842 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 14:56:26.762842 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 14:56:26.762842 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:56:26.762842 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:56:26.762842 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:56:26.762842 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:56:26.762842 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:56:26.762842 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:56:26.762842 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:56:26.762842 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 26 14:56:26.762842 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:56:26.762842 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 26 14:56:26.762842 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:56:26.762842 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:56:26.763720 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:56:26.763720 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 26 14:56:26.763720 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:56:26.763720 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 26 14:56:26.763720 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 14:56:26.763720 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 14:56:26.763720 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 14:56:26.763720 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 14:56:26.763720 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 14:56:26.763720 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 14:56:26.763720 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 14:56:26.763720 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 26 14:56:26.763720 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 540/1097] 10.222.0.46 () {56 vars in 1096 bytes} [Thu May 26 14:56:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 14:56:26.769123 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd0d7a2-4da0-4d3b-8e7d-e3186f71a8d2 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:56:26.789279 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e712e9-b0ea-442b-bb0a-5fc60c742794 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:56:26.789975 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e712e9-b0ea-442b-bb0a-5fc60c742794 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:26.790262 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e712e9-b0ea-442b-bb0a-5fc60c742794 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:26.790538 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e712e9-b0ea-442b-bb0a-5fc60c742794 None admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:26.791263 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e712e9-b0ea-442b-bb0a-5fc60c742794 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:56:26.791572 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e712e9-b0ea-442b-bb0a-5fc60c742794 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:56:26.792931 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e712e9-b0ea-442b-bb0a-5fc60c742794 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:56:26.804240 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 558/1098] 10.222.0.46 () {56 vars in 1109 bytes} [Thu May 26 14:56:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 14:56:26.810325 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e712e9-b0ea-442b-bb0a-5fc60c742794 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:56:26.824751 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:56:26.875447 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd0d7a2-4da0-4d3b-8e7d-e3186f71a8d2 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:56:26.909377 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f63c858-bda6-462a-bf0a-cf4267dd72d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:56:26.910072 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f63c858-bda6-462a-bf0a-cf4267dd72d8 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:26.910623 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f63c858-bda6-462a-bf0a-cf4267dd72d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:26.910756 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e712e9-b0ea-442b-bb0a-5fc60c742794 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:56:26.910850 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f63c858-bda6-462a-bf0a-cf4267dd72d8 None admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:26.911371 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f63c858-bda6-462a-bf0a-cf4267dd72d8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:56:26.911677 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f63c858-bda6-462a-bf0a-cf4267dd72d8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:56:26.913065 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f63c858-bda6-462a-bf0a-cf4267dd72d8 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:56:26.921712 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 541/1099] 10.222.0.46 () {56 vars in 1099 bytes} [Thu May 26 14:56:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 14:56:26.924547 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:56:26.927835 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f63c858-bda6-462a-bf0a-cf4267dd72d8 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:56:26.946174 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39a532c-ceda-4eb3-8a3f-8ce9b847eee3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:56:26.946994 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39a532c-ceda-4eb3-8a3f-8ce9b847eee3 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:26.947281 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39a532c-ceda-4eb3-8a3f-8ce9b847eee3 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:26.947556 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39a532c-ceda-4eb3-8a3f-8ce9b847eee3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:26.954987 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39a532c-ceda-4eb3-8a3f-8ce9b847eee3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:56:26.956524 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39a532c-ceda-4eb3-8a3f-8ce9b847eee3 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:56:26.971648 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-a39a532c-ceda-4eb3-8a3f-8ce9b847eee3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 14:56:26.971648 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:56:26.971648 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:56:26.971648 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:56:26.971648 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:56:26.971648 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:56:26.971648 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:56:26.971648 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:56:26.971648 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:56:26.971648 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:56:26.971648 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:56:26.971648 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:56:26.971648 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 14:56:26.971648 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 14:56:26.971648 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 14:56:26.972655 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 14:56:26.972655 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:56:26.972655 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:56:26.972655 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:56:26.972655 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:56:26.972655 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:56:26.972655 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:56:26.972655 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:56:26.972655 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 26 14:56:26.972655 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:56:26.972655 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 26 14:56:26.972655 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:56:26.972655 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:56:26.972655 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:56:26.972655 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 26 14:56:26.972655 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:56:26.972655 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 26 14:56:26.973587 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 14:56:26.973587 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 14:56:26.973587 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 14:56:26.973587 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 14:56:26.973587 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 14:56:26.973587 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 14:56:26.973587 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 14:56:26.973587 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 26 14:56:26.973587 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 559/1100] 10.222.0.46 () {56 vars in 1111 bytes} [Thu May 26 14:56:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 14:56:26.979445 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39a532c-ceda-4eb3-8a3f-8ce9b847eee3 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:56:27.034558 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f63c858-bda6-462a-bf0a-cf4267dd72d8 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:56:27.048441 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:56:27.070598 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17511ce-d6f9-4341-bc2a-cf0e33d7cdb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:56:27.071481 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17511ce-d6f9-4341-bc2a-cf0e33d7cdb6 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:27.071762 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17511ce-d6f9-4341-bc2a-cf0e33d7cdb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:27.072036 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17511ce-d6f9-4341-bc2a-cf0e33d7cdb6 None admin] PATH_INFO: `/v3/users/af259dcfa94146b5834c532e1496cfcf` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:27.073265 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17511ce-d6f9-4341-bc2a-cf0e33d7cdb6 None admin] RBAC: Authorizing `identity:get_user(user_id=af259dcfa94146b5834c532e1496cfcf)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:56:27.074741 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17511ce-d6f9-4341-bc2a-cf0e33d7cdb6 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:56:27.081114 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 542/1101] 10.222.0.46 () {56 vars in 1177 bytes} [Thu May 26 14:56:26 2022] GET /identity/v3/users/af259dcfa94146b5834c532e1496cfcf => generated 285 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:56:27.082339 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17511ce-d6f9-4341-bc2a-cf0e33d7cdb6 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:56:27.084148 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39a532c-ceda-4eb3-8a3f-8ce9b847eee3 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:56:27.119700 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de39511-c774-41f4-94fa-c1dffe619fe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:56:27.120365 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de39511-c774-41f4-94fa-c1dffe619fe0 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:27.120648 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de39511-c774-41f4-94fa-c1dffe619fe0 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:27.120924 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de39511-c774-41f4-94fa-c1dffe619fe0 None admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:27.121746 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de39511-c774-41f4-94fa-c1dffe619fe0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:56:27.122055 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de39511-c774-41f4-94fa-c1dffe619fe0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:56:27.123486 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de39511-c774-41f4-94fa-c1dffe619fe0 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:56:27.135504 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 560/1102] 10.222.0.46 () {56 vars in 1148 bytes} [Thu May 26 14:56:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 14:56:27.141558 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de39511-c774-41f4-94fa-c1dffe619fe0 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:56:27.175794 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:56:27.181256 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17511ce-d6f9-4341-bc2a-cf0e33d7cdb6 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:56:27.214961 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567c5818-2ae3-4817-9a30-d681a24bd9f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:56:27.215622 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567c5818-2ae3-4817-9a30-d681a24bd9f0 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:27.215908 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567c5818-2ae3-4817-9a30-d681a24bd9f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:27.216192 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567c5818-2ae3-4817-9a30-d681a24bd9f0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:27.243050 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de39511-c774-41f4-94fa-c1dffe619fe0 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:56:27.264202 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:56:27.277686 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2212476c-da82-45b9-a0cc-c705d8fee7ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:56:27.278347 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2212476c-da82-45b9-a0cc-c705d8fee7ec None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:27.278632 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2212476c-da82-45b9-a0cc-c705d8fee7ec None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:27.278909 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2212476c-da82-45b9-a0cc-c705d8fee7ec None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:27.279739 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2212476c-da82-45b9-a0cc-c705d8fee7ec None admin] RBAC: Adding query filter params (role.id=3593816f89a648809d5d7c9160fb87dc, scope.project.id=a17fc8227d4c4da6b2847b350e912ee3, user.id=d6639353c70449d1b3d10cd6487c3b00) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:56:27.280049 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2212476c-da82-45b9-a0cc-c705d8fee7ec None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:56:27.281496 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2212476c-da82-45b9-a0cc-c705d8fee7ec None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:56:27.285237 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567c5818-2ae3-4817-9a30-d681a24bd9f0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:56:27.286729 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567c5818-2ae3-4817-9a30-d681a24bd9f0 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:56:27.287378 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 561/1103] 10.222.0.46 () {56 vars in 1374 bytes} [Thu May 26 14:56:27 2022] GET /identity/v3/role_assignments?user.id=d6639353c70449d1b3d10cd6487c3b00&scope.project.id=a17fc8227d4c4da6b2847b350e912ee3&role.id=3593816f89a648809d5d7c9160fb87dc => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 14:56:27.397084 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 543/1104] 10.222.0.46 () {58 vars in 1282 bytes} [Thu May 26 14:56:27 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 26 14:56:27.403482 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2212476c-da82-45b9-a0cc-c705d8fee7ec None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:56:27.506871 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:56:27.512080 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2212476c-da82-45b9-a0cc-c705d8fee7ec None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:56:27.547220 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f54fc19-891e-4d6c-bcb4-b9309cad2b50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:56:27.548091 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f54fc19-891e-4d6c-bcb4-b9309cad2b50 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:27.548377 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f54fc19-891e-4d6c-bcb4-b9309cad2b50 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:27.548656 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f54fc19-891e-4d6c-bcb4-b9309cad2b50 None admin] PATH_INFO: `/v3/projects/5bcc4bc1b6ef4339b66e4943d0597e03` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:27.549915 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f54fc19-891e-4d6c-bcb4-b9309cad2b50 None admin] RBAC: Authorizing `identity:get_project(project_id=5bcc4bc1b6ef4339b66e4943d0597e03)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:56:27.551356 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f54fc19-891e-4d6c-bcb4-b9309cad2b50 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:56:27.553216 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 562/1105] 10.222.0.46 () {56 vars in 1186 bytes} [Thu May 26 14:56:27 2022] GET /identity/v3/projects/5bcc4bc1b6ef4339b66e4943d0597e03 => generated 311 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:56:27.559967 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567c5818-2ae3-4817-9a30-d681a24bd9f0 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:56:27.646659 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:56:27.667889 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567c5818-2ae3-4817-9a30-d681a24bd9f0 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:56:27.705898 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac81dee-a6d1-4fbe-bb69-54c95765178d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:56:27.706575 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac81dee-a6d1-4fbe-bb69-54c95765178d None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:27.706862 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac81dee-a6d1-4fbe-bb69-54c95765178d None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:27.707146 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac81dee-a6d1-4fbe-bb69-54c95765178d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:27.707984 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac81dee-a6d1-4fbe-bb69-54c95765178d None admin] RBAC: Adding query filter params (role.id=a2e551773cbd4df084020676865b8222, scope.project.id=5bcc4bc1b6ef4339b66e4943d0597e03, user.id=af259dcfa94146b5834c532e1496cfcf) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:56:27.708293 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac81dee-a6d1-4fbe-bb69-54c95765178d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:56:27.709814 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac81dee-a6d1-4fbe-bb69-54c95765178d None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:56:27.716120 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 544/1106] 10.222.0.46 () {56 vars in 1374 bytes} [Thu May 26 14:56:27 2022] GET /identity/v3/role_assignments?user.id=af259dcfa94146b5834c532e1496cfcf&scope.project.id=5bcc4bc1b6ef4339b66e4943d0597e03&role.id=a2e551773cbd4df084020676865b8222 => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:56:29.941238 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976f3f89-a4a2-4443-a7d9-64f32a52f0c8 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:29.941777 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976f3f89-a4a2-4443-a7d9-64f32a52f0c8 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:29.941777 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976f3f89-a4a2-4443-a7d9-64f32a52f0c8 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:29.943103 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 563/1107] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 14:56:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 14:56:29.948901 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dd8a12-94d3-4471-bdf7-66e85d55306c None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:29.949302 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dd8a12-94d3-4471-bdf7-66e85d55306c None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:29.949486 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dd8a12-94d3-4471-bdf7-66e85d55306c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:29.966654 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-58dd8a12-94d3-4471-bdf7-66e85d55306c None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:56:29.967858 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-58dd8a12-94d3-4471-bdf7-66e85d55306c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:56:30.032497 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 545/1108] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 14:56:29 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 14:56:30.056759 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976f3f89-a4a2-4443-a7d9-64f32a52f0c8 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:56:30.156037 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976f3f89-a4a2-4443-a7d9-64f32a52f0c8 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:56:30.188796 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d28b50-b387-4135-a290-56bcbb22b522 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:56:30.189607 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d28b50-b387-4135-a290-56bcbb22b522 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:30.189888 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d28b50-b387-4135-a290-56bcbb22b522 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:30.190159 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d28b50-b387-4135-a290-56bcbb22b522 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:30.191362 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d28b50-b387-4135-a290-56bcbb22b522 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:56:30.192722 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d28b50-b387-4135-a290-56bcbb22b522 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:56:30.194361 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 564/1109] 10.222.0.46 () {56 vars in 1108 bytes} [Thu May 26 14:56:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:56:30.200225 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58dd8a12-94d3-4471-bdf7-66e85d55306c None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:56:30.272066 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:56:30.297554 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58dd8a12-94d3-4471-bdf7-66e85d55306c None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:56:30.338596 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d49c92-9547-4993-9729-c67e5245007a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:56:30.339274 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d49c92-9547-4993-9729-c67e5245007a None admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:30.339560 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d49c92-9547-4993-9729-c67e5245007a None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:30.339840 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d49c92-9547-4993-9729-c67e5245007a None admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:30.340548 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d49c92-9547-4993-9729-c67e5245007a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:56:30.342021 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d49c92-9547-4993-9729-c67e5245007a None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:56:30.362967 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 546/1110] 10.222.0.46 () {60 vars in 1132 bytes} [Thu May 26 14:56:30 2022] POST /identity/v3/users => generated 286 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 14:56:32.602152 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58508810-93fb-41e9-8efb-4419746b22d6 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:32.602717 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58508810-93fb-41e9-8efb-4419746b22d6 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:32.602717 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58508810-93fb-41e9-8efb-4419746b22d6 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:32.604052 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 565/1111] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 14:56:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 14:56:32.609474 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c70d306-614b-4632-aa35-88c0bbf3f842 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:32.609939 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c70d306-614b-4632-aa35-88c0bbf3f842 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:32.610030 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c70d306-614b-4632-aa35-88c0bbf3f842 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:32.627504 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-0c70d306-614b-4632-aa35-88c0bbf3f842 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:56:32.628952 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-0c70d306-614b-4632-aa35-88c0bbf3f842 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:56:32.693771 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 547/1112] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 14:56:32 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 14:56:32.717015 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58508810-93fb-41e9-8efb-4419746b22d6 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:56:32.751735 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:56:32.817859 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58508810-93fb-41e9-8efb-4419746b22d6 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:56:32.850464 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f841b3ab-1192-4cad-9bb4-702747ec2815 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:56:32.851859 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f841b3ab-1192-4cad-9bb4-702747ec2815 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:32.852138 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f841b3ab-1192-4cad-9bb4-702747ec2815 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:32.852460 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f841b3ab-1192-4cad-9bb4-702747ec2815 None admin] PATH_INFO: `/v3/roles/160115b2143c4b389d39cd93f43e1d01` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:32.853928 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f841b3ab-1192-4cad-9bb4-702747ec2815 None admin] RBAC: Authorizing `identity:get_role(role_id=160115b2143c4b389d39cd93f43e1d01)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:56:32.855302 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f841b3ab-1192-4cad-9bb4-702747ec2815 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:56:32.856705 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 566/1113] 10.222.0.46 () {56 vars in 1177 bytes} [Thu May 26 14:56:32 2022] GET /identity/v3/roles/160115b2143c4b389d39cd93f43e1d01 => generated 223 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:56:32.862754 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c70d306-614b-4632-aa35-88c0bbf3f842 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:56:32.923041 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:56:32.964421 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c70d306-614b-4632-aa35-88c0bbf3f842 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:56:32.999188 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa35cc2-9d88-4f47-a0ad-16979575cb86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:56:33.000078 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa35cc2-9d88-4f47-a0ad-16979575cb86 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:33.000361 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa35cc2-9d88-4f47-a0ad-16979575cb86 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:33.000640 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa35cc2-9d88-4f47-a0ad-16979575cb86 None admin] PATH_INFO: `/v3/users/36459196a46f4443a7d07463baeb785f` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:33.014848 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa35cc2-9d88-4f47-a0ad-16979575cb86 None admin] RBAC: Authorizing `identity:get_user(user_id=36459196a46f4443a7d07463baeb785f)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:56:33.016300 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa35cc2-9d88-4f47-a0ad-16979575cb86 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:56:33.018066 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 548/1114] 10.222.0.46 () {56 vars in 1177 bytes} [Thu May 26 14:56:32 2022] GET /identity/v3/users/36459196a46f4443a7d07463baeb785f => generated 286 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:56:33.023844 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f841b3ab-1192-4cad-9bb4-702747ec2815 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:56:33.121889 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f841b3ab-1192-4cad-9bb4-702747ec2815 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:56:33.123339 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:56:33.156501 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25eeac1a-e1c3-46d6-a02a-40daca6cc9d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:56:33.157149 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25eeac1a-e1c3-46d6-a02a-40daca6cc9d5 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:33.157431 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25eeac1a-e1c3-46d6-a02a-40daca6cc9d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:33.157737 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25eeac1a-e1c3-46d6-a02a-40daca6cc9d5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:33.223475 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25eeac1a-e1c3-46d6-a02a-40daca6cc9d5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:56:33.224922 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25eeac1a-e1c3-46d6-a02a-40daca6cc9d5 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:56:33.320373 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 567/1115] 10.222.0.46 () {58 vars in 1282 bytes} [Thu May 26 14:56:33 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 26 14:56:33.326267 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa35cc2-9d88-4f47-a0ad-16979575cb86 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:56:33.425293 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa35cc2-9d88-4f47-a0ad-16979575cb86 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:56:33.442141 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:56:33.458610 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a4bd4d-ce60-456c-b9c0-6ed3f06e91a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:56:33.459477 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a4bd4d-ce60-456c-b9c0-6ed3f06e91a7 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:33.459760 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a4bd4d-ce60-456c-b9c0-6ed3f06e91a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:33.460038 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a4bd4d-ce60-456c-b9c0-6ed3f06e91a7 None admin] PATH_INFO: `/v3/projects/5bcc4bc1b6ef4339b66e4943d0597e03` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:33.461765 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a4bd4d-ce60-456c-b9c0-6ed3f06e91a7 None admin] RBAC: Authorizing `identity:get_project(project_id=5bcc4bc1b6ef4339b66e4943d0597e03)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:56:33.463205 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a4bd4d-ce60-456c-b9c0-6ed3f06e91a7 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:56:33.464911 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 549/1116] 10.222.0.46 () {56 vars in 1186 bytes} [Thu May 26 14:56:33 2022] GET /identity/v3/projects/5bcc4bc1b6ef4339b66e4943d0597e03 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:56:33.471002 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25eeac1a-e1c3-46d6-a02a-40daca6cc9d5 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:56:33.473180 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:56:33.569347 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25eeac1a-e1c3-46d6-a02a-40daca6cc9d5 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:56:33.602981 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9deefba5-6a58-4990-9c80-520aa5d1264e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:56:33.603619 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9deefba5-6a58-4990-9c80-520aa5d1264e None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:33.603889 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9deefba5-6a58-4990-9c80-520aa5d1264e None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:33.604157 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9deefba5-6a58-4990-9c80-520aa5d1264e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:33.604982 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9deefba5-6a58-4990-9c80-520aa5d1264e None admin] RBAC: Adding query filter params (role.id=160115b2143c4b389d39cd93f43e1d01, scope.project.id=5bcc4bc1b6ef4339b66e4943d0597e03, user.id=36459196a46f4443a7d07463baeb785f) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:56:33.605286 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9deefba5-6a58-4990-9c80-520aa5d1264e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:56:33.606821 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9deefba5-6a58-4990-9c80-520aa5d1264e None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:56:33.612449 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 568/1117] 10.222.0.46 () {56 vars in 1374 bytes} [Thu May 26 14:56:33 2022] GET /identity/v3/role_assignments?user.id=36459196a46f4443a7d07463baeb785f&scope.project.id=5bcc4bc1b6ef4339b66e4943d0597e03&role.id=160115b2143c4b389d39cd93f43e1d01 => generated 259 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:56:35.803501 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a557f927-c0bd-4e69-9ca1-863023f1f8fe None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:35.804593 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a557f927-c0bd-4e69-9ca1-863023f1f8fe None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:35.804593 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a557f927-c0bd-4e69-9ca1-863023f1f8fe None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:35.806072 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 550/1118] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 14:56:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 14:56:35.811361 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f29a13b-dcc8-4eac-8a7e-c970e5a3db55 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:35.811760 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f29a13b-dcc8-4eac-8a7e-c970e5a3db55 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:35.811913 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f29a13b-dcc8-4eac-8a7e-c970e5a3db55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:35.828831 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-8f29a13b-dcc8-4eac-8a7e-c970e5a3db55 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:56:35.830039 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-8f29a13b-dcc8-4eac-8a7e-c970e5a3db55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:56:35.888460 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:56:35.896270 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 569/1119] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 14:56:35 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 14:56:35.919795 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a557f927-c0bd-4e69-9ca1-863023f1f8fe None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:56:36.020188 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a557f927-c0bd-4e69-9ca1-863023f1f8fe None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:56:36.056032 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fac69a-e252-44f7-901f-65a7b60e64a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:56:36.057098 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fac69a-e252-44f7-901f-65a7b60e64a5 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:36.057377 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fac69a-e252-44f7-901f-65a7b60e64a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:36.057674 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fac69a-e252-44f7-901f-65a7b60e64a5 None admin] PATH_INFO: `/v3/roles/160115b2143c4b389d39cd93f43e1d01` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:36.059739 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5fac69a-e252-44f7-901f-65a7b60e64a5 None admin] RBAC: Authorizing `identity:get_role(role_id=160115b2143c4b389d39cd93f43e1d01)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:56:36.061633 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5fac69a-e252-44f7-901f-65a7b60e64a5 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:56:36.063020 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 551/1120] 10.222.0.46 () {56 vars in 1177 bytes} [Thu May 26 14:56:35 2022] GET /identity/v3/roles/160115b2143c4b389d39cd93f43e1d01 => generated 223 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:56:36.069000 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f29a13b-dcc8-4eac-8a7e-c970e5a3db55 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:56:36.174497 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f29a13b-dcc8-4eac-8a7e-c970e5a3db55 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:56:36.207355 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469256c5-b684-4127-9e99-d7b0454d20f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:56:36.208218 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469256c5-b684-4127-9e99-d7b0454d20f9 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:36.208499 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469256c5-b684-4127-9e99-d7b0454d20f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:36.208771 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469256c5-b684-4127-9e99-d7b0454d20f9 None admin] PATH_INFO: `/v3/users/36459196a46f4443a7d07463baeb785f` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:36.209979 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469256c5-b684-4127-9e99-d7b0454d20f9 None admin] RBAC: Authorizing `identity:get_user(user_id=36459196a46f4443a7d07463baeb785f)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:56:36.211911 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469256c5-b684-4127-9e99-d7b0454d20f9 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:56:36.213558 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 570/1121] 10.222.0.46 () {56 vars in 1177 bytes} [Thu May 26 14:56:36 2022] GET /identity/v3/users/36459196a46f4443a7d07463baeb785f => generated 286 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:56:36.219440 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fac69a-e252-44f7-901f-65a7b60e64a5 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:56:36.273281 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:56:36.316139 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fac69a-e252-44f7-901f-65a7b60e64a5 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:56:36.348821 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933c3dbf-1f20-4319-b857-e952134d8c5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:56:36.349718 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933c3dbf-1f20-4319-b857-e952134d8c5c None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:36.350001 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933c3dbf-1f20-4319-b857-e952134d8c5c None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:36.350273 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933c3dbf-1f20-4319-b857-e952134d8c5c None admin] PATH_INFO: `/v3/projects/5bcc4bc1b6ef4339b66e4943d0597e03` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:36.352041 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933c3dbf-1f20-4319-b857-e952134d8c5c None admin] RBAC: Authorizing `identity:get_project(project_id=5bcc4bc1b6ef4339b66e4943d0597e03)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:56:36.353496 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933c3dbf-1f20-4319-b857-e952134d8c5c None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:56:36.355196 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 552/1122] 10.222.0.46 () {56 vars in 1186 bytes} [Thu May 26 14:56:36 2022] GET /identity/v3/projects/5bcc4bc1b6ef4339b66e4943d0597e03 => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:56:36.361361 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469256c5-b684-4127-9e99-d7b0454d20f9 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:56:36.423899 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:56:36.463121 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469256c5-b684-4127-9e99-d7b0454d20f9 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:56:36.496269 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc7088e-9154-46f0-a35b-6fb9d0d33626 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:56:36.497157 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc7088e-9154-46f0-a35b-6fb9d0d33626 None admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:36.497441 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc7088e-9154-46f0-a35b-6fb9d0d33626 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:36.497750 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc7088e-9154-46f0-a35b-6fb9d0d33626 None admin] PATH_INFO: `/v3/projects/5bcc4bc1b6ef4339b66e4943d0597e03/users/36459196a46f4443a7d07463baeb785f/roles/160115b2143c4b389d39cd93f43e1d01` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:36.499956 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc7088e-9154-46f0-a35b-6fb9d0d33626 None admin] RBAC: Authorizing `identity:create_grant(project_id=5bcc4bc1b6ef4339b66e4943d0597e03, user_id=36459196a46f4443a7d07463baeb785f, role_id=160115b2143c4b389d39cd93f43e1d01)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:56:36.501389 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc7088e-9154-46f0-a35b-6fb9d0d33626 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:56:36.514722 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 571/1123] 10.222.0.46 () {58 vars in 1439 bytes} [Thu May 26 14:56:36 2022] PUT /identity/v3/projects/5bcc4bc1b6ef4339b66e4943d0597e03/users/36459196a46f4443a7d07463baeb785f/roles/160115b2143c4b389d39cd93f43e1d01 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 14:56:38.710877 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ec8f4e-2a01-4c62-890d-783273bb149a None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:38.711440 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ec8f4e-2a01-4c62-890d-783273bb149a None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:38.711440 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ec8f4e-2a01-4c62-890d-783273bb149a None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:38.712815 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 553/1124] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 14:56:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 14:56:38.718257 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b418972-4c00-4028-acf5-d2c4f5fe798c None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:38.718726 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b418972-4c00-4028-acf5-d2c4f5fe798c None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:38.718833 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b418972-4c00-4028-acf5-d2c4f5fe798c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:38.735165 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-7b418972-4c00-4028-acf5-d2c4f5fe798c None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:56:38.736346 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-7b418972-4c00-4028-acf5-d2c4f5fe798c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:56:38.802164 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 572/1125] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 14:56:38 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 14:56:38.827754 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ec8f4e-2a01-4c62-890d-783273bb149a None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:56:38.927066 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ec8f4e-2a01-4c62-890d-783273bb149a None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:56:38.949205 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:56:38.960806 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5530c9-9389-4350-ac1b-834ab0cb67e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:56:38.961692 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5530c9-9389-4350-ac1b-834ab0cb67e3 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:38.961979 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5530c9-9389-4350-ac1b-834ab0cb67e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:38.962257 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5530c9-9389-4350-ac1b-834ab0cb67e3 None admin] PATH_INFO: `/v3/roles/160115b2143c4b389d39cd93f43e1d01` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:38.963713 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5530c9-9389-4350-ac1b-834ab0cb67e3 None admin] RBAC: Authorizing `identity:get_role(role_id=160115b2143c4b389d39cd93f43e1d01)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:56:38.965091 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5530c9-9389-4350-ac1b-834ab0cb67e3 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:56:38.966518 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 554/1126] 10.222.0.46 () {56 vars in 1177 bytes} [Thu May 26 14:56:38 2022] GET /identity/v3/roles/160115b2143c4b389d39cd93f43e1d01 => generated 223 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:56:38.972313 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b418972-4c00-4028-acf5-d2c4f5fe798c None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:56:39.026893 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:56:39.072157 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b418972-4c00-4028-acf5-d2c4f5fe798c None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:56:39.105347 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b934de-214a-4914-9de8-c8a2b59c511b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:56:39.106252 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b934de-214a-4914-9de8-c8a2b59c511b None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:39.106533 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b934de-214a-4914-9de8-c8a2b59c511b None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:39.106809 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b934de-214a-4914-9de8-c8a2b59c511b None admin] PATH_INFO: `/v3/users/36459196a46f4443a7d07463baeb785f` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:39.108028 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b934de-214a-4914-9de8-c8a2b59c511b None admin] RBAC: Authorizing `identity:get_user(user_id=36459196a46f4443a7d07463baeb785f)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:56:39.109543 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b934de-214a-4914-9de8-c8a2b59c511b None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:56:39.111196 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 573/1127] 10.222.0.46 () {56 vars in 1177 bytes} [Thu May 26 14:56:38 2022] GET /identity/v3/users/36459196a46f4443a7d07463baeb785f => generated 286 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:56:39.117639 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5530c9-9389-4350-ac1b-834ab0cb67e3 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:56:39.218173 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5530c9-9389-4350-ac1b-834ab0cb67e3 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:56:39.252707 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2288656a-6d64-4a55-841c-855d7bec1ff9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:56:39.253355 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2288656a-6d64-4a55-841c-855d7bec1ff9 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:39.253927 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2288656a-6d64-4a55-841c-855d7bec1ff9 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:39.254205 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2288656a-6d64-4a55-841c-855d7bec1ff9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:39.309324 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:56:39.319562 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2288656a-6d64-4a55-841c-855d7bec1ff9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:56:39.321014 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2288656a-6d64-4a55-841c-855d7bec1ff9 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:56:39.417354 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 555/1128] 10.222.0.46 () {58 vars in 1282 bytes} [Thu May 26 14:56:39 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 26 14:56:39.423323 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b934de-214a-4914-9de8-c8a2b59c511b None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:56:39.521366 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b934de-214a-4914-9de8-c8a2b59c511b None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:56:39.544668 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:56:39.556427 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218a0a52-e8ae-475f-981b-42c7cf2906a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:56:39.557290 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218a0a52-e8ae-475f-981b-42c7cf2906a3 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:39.557603 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218a0a52-e8ae-475f-981b-42c7cf2906a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:39.557877 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218a0a52-e8ae-475f-981b-42c7cf2906a3 None admin] PATH_INFO: `/v3/projects/5bcc4bc1b6ef4339b66e4943d0597e03` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:39.559142 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218a0a52-e8ae-475f-981b-42c7cf2906a3 None admin] RBAC: Authorizing `identity:get_project(project_id=5bcc4bc1b6ef4339b66e4943d0597e03)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:56:39.560581 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218a0a52-e8ae-475f-981b-42c7cf2906a3 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:56:39.562358 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 574/1129] 10.222.0.46 () {56 vars in 1186 bytes} [Thu May 26 14:56:39 2022] GET /identity/v3/projects/5bcc4bc1b6ef4339b66e4943d0597e03 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:56:39.568421 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2288656a-6d64-4a55-841c-855d7bec1ff9 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:56:39.657461 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:56:39.667160 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2288656a-6d64-4a55-841c-855d7bec1ff9 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:56:39.699859 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb18561-5902-49d3-82a5-ee4df0c2b87b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:56:39.700567 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb18561-5902-49d3-82a5-ee4df0c2b87b None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:39.700848 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb18561-5902-49d3-82a5-ee4df0c2b87b None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:39.701121 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb18561-5902-49d3-82a5-ee4df0c2b87b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:39.701969 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb18561-5902-49d3-82a5-ee4df0c2b87b None admin] RBAC: Adding query filter params (role.id=160115b2143c4b389d39cd93f43e1d01, scope.project.id=5bcc4bc1b6ef4339b66e4943d0597e03, user.id=36459196a46f4443a7d07463baeb785f) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:56:39.702277 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb18561-5902-49d3-82a5-ee4df0c2b87b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:56:39.703704 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb18561-5902-49d3-82a5-ee4df0c2b87b None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:56:39.709696 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 556/1130] 10.222.0.46 () {56 vars in 1374 bytes} [Thu May 26 14:56:39 2022] GET /identity/v3/role_assignments?user.id=36459196a46f4443a7d07463baeb785f&scope.project.id=5bcc4bc1b6ef4339b66e4943d0597e03&role.id=160115b2143c4b389d39cd93f43e1d01 => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:56:41.907872 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f32044e-4091-43dd-8034-ffaeffa19b6e None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:41.908369 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f32044e-4091-43dd-8034-ffaeffa19b6e None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:41.908369 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f32044e-4091-43dd-8034-ffaeffa19b6e None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:41.909766 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 575/1131] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 14:56:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 14:56:41.914905 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663acb78-b901-454c-a67e-c543f713c380 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:41.915272 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663acb78-b901-454c-a67e-c543f713c380 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:41.915465 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663acb78-b901-454c-a67e-c543f713c380 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:41.931745 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-663acb78-b901-454c-a67e-c543f713c380 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:56:41.932936 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-663acb78-b901-454c-a67e-c543f713c380 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:56:41.997537 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 557/1132] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 14:56:41 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 14:56:42.021371 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f32044e-4091-43dd-8034-ffaeffa19b6e None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:56:42.122493 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f32044e-4091-43dd-8034-ffaeffa19b6e None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:56:42.155611 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13608c7f-1212-4589-8fa3-12472ab49740 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:56:42.156393 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13608c7f-1212-4589-8fa3-12472ab49740 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:42.156677 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13608c7f-1212-4589-8fa3-12472ab49740 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:42.156954 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13608c7f-1212-4589-8fa3-12472ab49740 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:42.158190 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13608c7f-1212-4589-8fa3-12472ab49740 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:56:42.159577 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13608c7f-1212-4589-8fa3-12472ab49740 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:56:42.161199 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 576/1133] 10.222.0.46 () {56 vars in 1108 bytes} [Thu May 26 14:56:42 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:56:42.167548 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663acb78-b901-454c-a67e-c543f713c380 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:56:42.247236 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:56:42.262542 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663acb78-b901-454c-a67e-c543f713c380 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:56:42.295518 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046fdbac-8acf-43eb-9813-19014d917cab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:56:42.296172 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046fdbac-8acf-43eb-9813-19014d917cab None admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:42.296454 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046fdbac-8acf-43eb-9813-19014d917cab None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:42.296727 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046fdbac-8acf-43eb-9813-19014d917cab None admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:42.297422 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046fdbac-8acf-43eb-9813-19014d917cab None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:56:42.298892 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046fdbac-8acf-43eb-9813-19014d917cab None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:56:42.325884 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 558/1134] 10.222.0.46 () {60 vars in 1141 bytes} [Thu May 26 14:56:42 2022] POST /identity/v3/projects => generated 311 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 14:56:44.560444 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74829c0-6bb6-4ca2-841b-4828746023e5 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:44.561046 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74829c0-6bb6-4ca2-841b-4828746023e5 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:44.561046 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74829c0-6bb6-4ca2-841b-4828746023e5 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:44.562408 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 577/1135] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 14:56:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 14:56:44.567684 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e058d1-cf6d-4484-8b68-29c5ad9ee420 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:44.568170 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e058d1-cf6d-4484-8b68-29c5ad9ee420 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:44.568271 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e058d1-cf6d-4484-8b68-29c5ad9ee420 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:44.585480 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-35e058d1-cf6d-4484-8b68-29c5ad9ee420 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:56:44.586691 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-35e058d1-cf6d-4484-8b68-29c5ad9ee420 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:56:44.654918 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 559/1136] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 14:56:44 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 14:56:44.680208 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74829c0-6bb6-4ca2-841b-4828746023e5 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:56:44.739800 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:56:44.784165 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74829c0-6bb6-4ca2-841b-4828746023e5 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:56:44.818024 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe5ea58-7ba7-46b5-a1cd-c93c1c1d0b3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:56:44.818741 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe5ea58-7ba7-46b5-a1cd-c93c1c1d0b3c None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:44.819538 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe5ea58-7ba7-46b5-a1cd-c93c1c1d0b3c None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:44.819813 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe5ea58-7ba7-46b5-a1cd-c93c1c1d0b3c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:44.820985 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe5ea58-7ba7-46b5-a1cd-c93c1c1d0b3c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:56:44.822445 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe5ea58-7ba7-46b5-a1cd-c93c1c1d0b3c None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:56:44.824057 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 578/1137] 10.222.0.46 () {56 vars in 1108 bytes} [Thu May 26 14:56:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:56:44.829926 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e058d1-cf6d-4484-8b68-29c5ad9ee420 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:56:44.894831 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:56:44.929165 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e058d1-cf6d-4484-8b68-29c5ad9ee420 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:56:44.962389 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6c2769-547a-4fb2-bc0e-cef3a4de096d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:56:44.963056 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6c2769-547a-4fb2-bc0e-cef3a4de096d None admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:44.963336 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6c2769-547a-4fb2-bc0e-cef3a4de096d None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:44.963608 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6c2769-547a-4fb2-bc0e-cef3a4de096d None admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:44.964316 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6c2769-547a-4fb2-bc0e-cef3a4de096d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:56:44.965770 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e6c2769-547a-4fb2-bc0e-cef3a4de096d None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:56:44.985316 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 560/1138] 10.222.0.46 () {60 vars in 1132 bytes} [Thu May 26 14:56:44 2022] POST /identity/v3/users => generated 286 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 14:56:47.283625 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4667dc2-0afb-4b39-aa4d-2786985e7221 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:47.284196 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4667dc2-0afb-4b39-aa4d-2786985e7221 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:47.284196 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4667dc2-0afb-4b39-aa4d-2786985e7221 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:47.285549 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 579/1139] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 14:56:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 14:56:47.290772 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dba2e0-0559-485e-bb85-959fce1543da None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:47.291163 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dba2e0-0559-485e-bb85-959fce1543da None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:47.291330 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dba2e0-0559-485e-bb85-959fce1543da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:47.307973 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-c5dba2e0-0559-485e-bb85-959fce1543da None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:56:47.309155 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-c5dba2e0-0559-485e-bb85-959fce1543da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:56:47.384177 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 561/1140] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 14:56:47 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 14:56:47.408173 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4667dc2-0afb-4b39-aa4d-2786985e7221 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:56:47.525058 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4667dc2-0afb-4b39-aa4d-2786985e7221 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:56:47.554136 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:56:47.563339 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b0fa06-6f38-476a-a53a-17a26c0c1ec0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:56:47.564142 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b0fa06-6f38-476a-a53a-17a26c0c1ec0 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:47.564422 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b0fa06-6f38-476a-a53a-17a26c0c1ec0 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:47.564693 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b0fa06-6f38-476a-a53a-17a26c0c1ec0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:47.575608 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b0fa06-6f38-476a-a53a-17a26c0c1ec0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 14:56:47.583303 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-53b0fa06-6f38-476a-a53a-17a26c0c1ec0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 14:56:47.583303 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:56:47.583303 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:56:47.583303 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:56:47.583303 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:56:47.583303 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:56:47.583303 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:56:47.583303 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:56:47.583303 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:56:47.583303 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:56:47.583303 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:56:47.583303 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:56:47.583303 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 14:56:47.583303 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 14:56:47.583303 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 14:56:47.583303 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 14:56:47.584676 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 14:56:47.584676 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 14:56:47.584676 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 14:56:47.584676 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 14:56:47.584676 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:56:47.584676 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:56:47.584676 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:56:47.584676 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:56:47.584676 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:56:47.584676 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:56:47.584676 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:56:47.584676 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 26 14:56:47.584676 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:56:47.584676 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 26 14:56:47.584676 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:56:47.584676 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:56:47.585907 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:56:47.585907 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 26 14:56:47.585907 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:56:47.585907 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 26 14:56:47.585907 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 14:56:47.585907 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 14:56:47.585907 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 14:56:47.585907 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 14:56:47.585907 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 14:56:47.585907 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 14:56:47.585907 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 14:56:47.585907 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 26 14:56:47.585907 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 580/1141] 10.222.0.46 () {56 vars in 1096 bytes} [Thu May 26 14:56:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:56:47.590630 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dba2e0-0559-485e-bb85-959fce1543da None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:56:47.654634 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:56:47.707801 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dba2e0-0559-485e-bb85-959fce1543da None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:56:47.742465 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d0ae6c-9f84-48f4-86b0-e26d7945cde0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:56:47.743126 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d0ae6c-9f84-48f4-86b0-e26d7945cde0 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:47.743405 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d0ae6c-9f84-48f4-86b0-e26d7945cde0 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:47.743678 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d0ae6c-9f84-48f4-86b0-e26d7945cde0 None admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:47.744476 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d0ae6c-9f84-48f4-86b0-e26d7945cde0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:56:47.744855 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d0ae6c-9f84-48f4-86b0-e26d7945cde0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:56:47.746253 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d0ae6c-9f84-48f4-86b0-e26d7945cde0 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:56:47.755040 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 562/1142] 10.222.0.46 () {56 vars in 1099 bytes} [Thu May 26 14:56:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:56:47.763737 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b0fa06-6f38-476a-a53a-17a26c0c1ec0 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:56:47.876286 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b0fa06-6f38-476a-a53a-17a26c0c1ec0 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:56:47.912245 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21220c67-dbf1-4bb7-a4a6-67a90ea2045c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:56:47.913734 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21220c67-dbf1-4bb7-a4a6-67a90ea2045c None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:47.914057 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21220c67-dbf1-4bb7-a4a6-67a90ea2045c None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:47.914360 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21220c67-dbf1-4bb7-a4a6-67a90ea2045c None admin] PATH_INFO: `/v3/users/209453d7c80c4f44922d6300549194fc` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:47.930172 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21220c67-dbf1-4bb7-a4a6-67a90ea2045c None admin] RBAC: Authorizing `identity:get_user(user_id=209453d7c80c4f44922d6300549194fc)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:56:47.931772 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21220c67-dbf1-4bb7-a4a6-67a90ea2045c None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:56:47.933488 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 581/1143] 10.222.0.46 () {56 vars in 1177 bytes} [Thu May 26 14:56:47 2022] GET /identity/v3/users/209453d7c80c4f44922d6300549194fc => generated 286 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:56:47.939572 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d0ae6c-9f84-48f4-86b0-e26d7945cde0 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:56:48.046344 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d0ae6c-9f84-48f4-86b0-e26d7945cde0 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:56:48.059829 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:56:48.083275 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48dfa7f-ef27-4ebd-8bd2-b96d2a7f482b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:56:48.083931 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48dfa7f-ef27-4ebd-8bd2-b96d2a7f482b None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:48.084219 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48dfa7f-ef27-4ebd-8bd2-b96d2a7f482b None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:48.085144 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48dfa7f-ef27-4ebd-8bd2-b96d2a7f482b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:48.154770 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48dfa7f-ef27-4ebd-8bd2-b96d2a7f482b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:56:48.156168 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48dfa7f-ef27-4ebd-8bd2-b96d2a7f482b None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:56:48.260455 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 563/1144] 10.222.0.46 () {58 vars in 1282 bytes} [Thu May 26 14:56:47 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 26 14:56:48.266385 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21220c67-dbf1-4bb7-a4a6-67a90ea2045c None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:56:48.315196 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:56:48.371792 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21220c67-dbf1-4bb7-a4a6-67a90ea2045c None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:56:48.409317 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac45a26-10ce-4f2c-863b-1cfe3e38b65b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:56:48.410100 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac45a26-10ce-4f2c-863b-1cfe3e38b65b None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:48.410397 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac45a26-10ce-4f2c-863b-1cfe3e38b65b None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:48.410672 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac45a26-10ce-4f2c-863b-1cfe3e38b65b None admin] PATH_INFO: `/v3/projects/3fa285bf3cf347b1bface45a1758ad7f` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:48.411905 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac45a26-10ce-4f2c-863b-1cfe3e38b65b None admin] RBAC: Authorizing `identity:get_project(project_id=3fa285bf3cf347b1bface45a1758ad7f)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:56:48.413442 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac45a26-10ce-4f2c-863b-1cfe3e38b65b None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:56:48.415184 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 582/1145] 10.222.0.46 () {56 vars in 1186 bytes} [Thu May 26 14:56:48 2022] GET /identity/v3/projects/3fa285bf3cf347b1bface45a1758ad7f => generated 311 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:56:48.421422 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48dfa7f-ef27-4ebd-8bd2-b96d2a7f482b None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:56:48.442098 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:56:48.531656 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48dfa7f-ef27-4ebd-8bd2-b96d2a7f482b None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:56:48.571180 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8bd341-601c-4407-b859-8732382f752b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:56:48.571878 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8bd341-601c-4407-b859-8732382f752b None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:48.572173 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8bd341-601c-4407-b859-8732382f752b None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:48.572451 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8bd341-601c-4407-b859-8732382f752b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:48.573293 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8bd341-601c-4407-b859-8732382f752b None admin] RBAC: Adding query filter params (role.id=a2e551773cbd4df084020676865b8222, scope.project.id=3fa285bf3cf347b1bface45a1758ad7f, user.id=209453d7c80c4f44922d6300549194fc) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:56:48.573639 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8bd341-601c-4407-b859-8732382f752b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:56:48.575110 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8bd341-601c-4407-b859-8732382f752b None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:56:48.580906 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 564/1146] 10.222.0.46 () {56 vars in 1374 bytes} [Thu May 26 14:56:48 2022] GET /identity/v3/role_assignments?user.id=209453d7c80c4f44922d6300549194fc&scope.project.id=3fa285bf3cf347b1bface45a1758ad7f&role.id=a2e551773cbd4df084020676865b8222 => generated 259 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:56:50.818276 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816f92e1-fd45-46e3-9f4a-a5408ac5bea2 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:50.819312 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816f92e1-fd45-46e3-9f4a-a5408ac5bea2 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:50.819421 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816f92e1-fd45-46e3-9f4a-a5408ac5bea2 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:50.820702 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 583/1147] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 14:56:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 14:56:50.826178 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e3ca90-af96-4880-a315-29822fcafefb None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:50.826661 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e3ca90-af96-4880-a315-29822fcafefb None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:50.826769 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e3ca90-af96-4880-a315-29822fcafefb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:50.844598 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-f6e3ca90-af96-4880-a315-29822fcafefb None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:56:50.845812 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-f6e3ca90-af96-4880-a315-29822fcafefb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:56:50.919512 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 565/1148] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 14:56:50 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 26 14:56:50.941030 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816f92e1-fd45-46e3-9f4a-a5408ac5bea2 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:56:51.047560 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816f92e1-fd45-46e3-9f4a-a5408ac5bea2 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:56:51.061132 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:56:51.086431 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d177353a-e7ee-41d8-a9d5-e92100627326 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:56:51.087224 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d177353a-e7ee-41d8-a9d5-e92100627326 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:51.087506 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d177353a-e7ee-41d8-a9d5-e92100627326 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:51.087782 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d177353a-e7ee-41d8-a9d5-e92100627326 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:51.098358 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d177353a-e7ee-41d8-a9d5-e92100627326 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 14:56:51.106202 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-d177353a-e7ee-41d8-a9d5-e92100627326 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 14:56:51.106202 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:56:51.106202 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:56:51.106202 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:56:51.106202 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:56:51.106202 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:56:51.106202 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:56:51.106202 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:56:51.106202 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:56:51.106202 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:56:51.106202 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:56:51.106202 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:56:51.106202 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 14:56:51.106202 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 14:56:51.106202 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 14:56:51.106202 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 14:56:51.107334 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 14:56:51.107334 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 14:56:51.107334 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 14:56:51.107334 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 14:56:51.107334 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:56:51.107334 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:56:51.107334 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:56:51.107334 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:56:51.107334 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:56:51.107334 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:56:51.107334 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:56:51.107334 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 26 14:56:51.107334 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:56:51.107334 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 26 14:56:51.107334 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:56:51.107334 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:56:51.108245 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:56:51.108245 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 26 14:56:51.108245 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:56:51.108245 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 26 14:56:51.108245 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 14:56:51.108245 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 14:56:51.108245 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 14:56:51.108245 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 14:56:51.108245 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 14:56:51.108245 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 14:56:51.108245 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 14:56:51.108245 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 26 14:56:51.108245 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 584/1149] 10.222.0.46 () {56 vars in 1096 bytes} [Thu May 26 14:56:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:56:51.114487 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e3ca90-af96-4880-a315-29822fcafefb None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:56:51.125185 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:56:51.221045 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6e3ca90-af96-4880-a315-29822fcafefb None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:56:51.257289 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ebb60f-08d8-490c-8027-28549f7c7450 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:56:51.257984 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ebb60f-08d8-490c-8027-28549f7c7450 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:51.258266 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ebb60f-08d8-490c-8027-28549f7c7450 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:51.258538 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ebb60f-08d8-490c-8027-28549f7c7450 None admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:51.259423 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ebb60f-08d8-490c-8027-28549f7c7450 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:56:51.259806 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ebb60f-08d8-490c-8027-28549f7c7450 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:56:51.261234 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ebb60f-08d8-490c-8027-28549f7c7450 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:56:51.270345 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 566/1150] 10.222.0.46 () {56 vars in 1099 bytes} [Thu May 26 14:56:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:56:51.276447 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d177353a-e7ee-41d8-a9d5-e92100627326 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:56:51.381156 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d177353a-e7ee-41d8-a9d5-e92100627326 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:56:51.416317 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e27fe67-2114-4673-87e8-74f7cfe3ea30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:56:51.417194 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e27fe67-2114-4673-87e8-74f7cfe3ea30 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:51.417502 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e27fe67-2114-4673-87e8-74f7cfe3ea30 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:51.417784 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e27fe67-2114-4673-87e8-74f7cfe3ea30 None admin] PATH_INFO: `/v3/users/209453d7c80c4f44922d6300549194fc` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:51.418978 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e27fe67-2114-4673-87e8-74f7cfe3ea30 None admin] RBAC: Authorizing `identity:get_user(user_id=209453d7c80c4f44922d6300549194fc)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:56:51.420427 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e27fe67-2114-4673-87e8-74f7cfe3ea30 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:56:51.422124 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 585/1151] 10.222.0.46 () {56 vars in 1177 bytes} [Thu May 26 14:56:51 2022] GET /identity/v3/users/209453d7c80c4f44922d6300549194fc => generated 286 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:56:51.428622 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ebb60f-08d8-490c-8027-28549f7c7450 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:56:51.500573 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:56:51.534220 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ebb60f-08d8-490c-8027-28549f7c7450 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:56:51.573822 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979e0d00-7d4c-413f-a9b4-d8d182365d16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:56:51.574999 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979e0d00-7d4c-413f-a9b4-d8d182365d16 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:51.575404 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979e0d00-7d4c-413f-a9b4-d8d182365d16 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:51.575822 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979e0d00-7d4c-413f-a9b4-d8d182365d16 None admin] PATH_INFO: `/v3/projects/3fa285bf3cf347b1bface45a1758ad7f` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:51.577909 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979e0d00-7d4c-413f-a9b4-d8d182365d16 None admin] RBAC: Authorizing `identity:get_project(project_id=3fa285bf3cf347b1bface45a1758ad7f)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:56:51.580605 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979e0d00-7d4c-413f-a9b4-d8d182365d16 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:56:51.583130 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 567/1152] 10.222.0.46 () {56 vars in 1186 bytes} [Thu May 26 14:56:51 2022] GET /identity/v3/projects/3fa285bf3cf347b1bface45a1758ad7f => generated 311 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:56:51.589046 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e27fe67-2114-4673-87e8-74f7cfe3ea30 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:56:51.635702 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:56:51.700417 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e27fe67-2114-4673-87e8-74f7cfe3ea30 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:56:51.734695 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f8fc59-a4ae-49ed-a7d7-70f1ea5dfc57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:56:51.735591 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f8fc59-a4ae-49ed-a7d7-70f1ea5dfc57 None admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:51.735876 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f8fc59-a4ae-49ed-a7d7-70f1ea5dfc57 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:51.736152 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f8fc59-a4ae-49ed-a7d7-70f1ea5dfc57 None admin] PATH_INFO: `/v3/projects/3fa285bf3cf347b1bface45a1758ad7f/users/209453d7c80c4f44922d6300549194fc/roles/a2e551773cbd4df084020676865b8222` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:51.738118 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f8fc59-a4ae-49ed-a7d7-70f1ea5dfc57 None admin] RBAC: Authorizing `identity:create_grant(project_id=3fa285bf3cf347b1bface45a1758ad7f, user_id=209453d7c80c4f44922d6300549194fc, role_id=a2e551773cbd4df084020676865b8222)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:56:51.739561 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f8fc59-a4ae-49ed-a7d7-70f1ea5dfc57 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:56:51.754873 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 586/1153] 10.222.0.46 () {58 vars in 1439 bytes} [Thu May 26 14:56:51 2022] PUT /identity/v3/projects/3fa285bf3cf347b1bface45a1758ad7f/users/209453d7c80c4f44922d6300549194fc/roles/a2e551773cbd4df084020676865b8222 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 14:56:54.005233 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04b01ff-a068-40ae-bd8b-47357e7925f1 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:54.005781 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04b01ff-a068-40ae-bd8b-47357e7925f1 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:54.005781 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04b01ff-a068-40ae-bd8b-47357e7925f1 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:54.007654 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 568/1154] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 14:56:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 14:56:54.013251 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ba9b82-e665-40cd-9807-f9c13d40dc0e None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:54.013771 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ba9b82-e665-40cd-9807-f9c13d40dc0e None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:54.013903 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ba9b82-e665-40cd-9807-f9c13d40dc0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:54.031732 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-96ba9b82-e665-40cd-9807-f9c13d40dc0e None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:56:54.032935 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-96ba9b82-e665-40cd-9807-f9c13d40dc0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:56:54.103996 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 587/1155] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 14:56:54 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 14:56:54.128022 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04b01ff-a068-40ae-bd8b-47357e7925f1 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:56:54.234205 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04b01ff-a068-40ae-bd8b-47357e7925f1 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:56:54.268992 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6784a780-389a-423b-b376-fb872dee0e00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:56:54.269727 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6784a780-389a-423b-b376-fb872dee0e00 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:54.270010 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6784a780-389a-423b-b376-fb872dee0e00 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:54.270284 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6784a780-389a-423b-b376-fb872dee0e00 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:54.276772 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:56:54.281098 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6784a780-389a-423b-b376-fb872dee0e00 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 14:56:54.288977 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-6784a780-389a-423b-b376-fb872dee0e00 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 14:56:54.288977 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:56:54.288977 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:56:54.288977 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:56:54.288977 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:56:54.288977 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:56:54.288977 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:56:54.288977 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:56:54.288977 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:56:54.288977 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:56:54.288977 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:56:54.288977 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:56:54.288977 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 14:56:54.288977 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 14:56:54.288977 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 14:56:54.288977 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 14:56:54.290423 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 14:56:54.290423 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 14:56:54.290423 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 14:56:54.290423 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 14:56:54.290423 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:56:54.290423 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:56:54.290423 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:56:54.290423 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:56:54.290423 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:56:54.290423 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:56:54.290423 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:56:54.290423 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 26 14:56:54.290423 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:56:54.290423 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 26 14:56:54.290423 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:56:54.290423 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:56:54.291539 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:56:54.291539 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 26 14:56:54.291539 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:56:54.291539 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 26 14:56:54.291539 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 14:56:54.291539 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 14:56:54.291539 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 14:56:54.291539 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 14:56:54.291539 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 14:56:54.291539 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 14:56:54.291539 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 14:56:54.291539 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 26 14:56:54.291539 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 569/1156] 10.222.0.46 () {56 vars in 1096 bytes} [Thu May 26 14:56:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:56:54.296654 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ba9b82-e665-40cd-9807-f9c13d40dc0e None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:56:54.328553 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:56:54.406692 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ba9b82-e665-40cd-9807-f9c13d40dc0e None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:56:54.444467 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3b5ad9-ea7a-4768-90d4-557c3e10c019 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:56:54.445196 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3b5ad9-ea7a-4768-90d4-557c3e10c019 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:54.445541 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3b5ad9-ea7a-4768-90d4-557c3e10c019 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:54.445841 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3b5ad9-ea7a-4768-90d4-557c3e10c019 None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:54.446677 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3b5ad9-ea7a-4768-90d4-557c3e10c019 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:56:54.447115 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3b5ad9-ea7a-4768-90d4-557c3e10c019 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:56:54.448501 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d3b5ad9-ea7a-4768-90d4-557c3e10c019 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:56:54.457405 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 588/1157] 10.222.0.46 () {56 vars in 1099 bytes} [Thu May 26 14:56:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 14:56:54.463280 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6784a780-389a-423b-b376-fb872dee0e00 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:56:54.566740 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6784a780-389a-423b-b376-fb872dee0e00 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:56:54.604521 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc8bfe8-0569-4054-9628-0ba4c4961912 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:56:54.605420 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc8bfe8-0569-4054-9628-0ba4c4961912 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:54.605737 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc8bfe8-0569-4054-9628-0ba4c4961912 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:54.606014 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc8bfe8-0569-4054-9628-0ba4c4961912 None admin] PATH_INFO: `/v3/users/209453d7c80c4f44922d6300549194fc` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:54.607230 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc8bfe8-0569-4054-9628-0ba4c4961912 None admin] RBAC: Authorizing `identity:get_user(user_id=209453d7c80c4f44922d6300549194fc)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:56:54.608683 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc8bfe8-0569-4054-9628-0ba4c4961912 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:56:54.610380 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 570/1158] 10.222.0.46 () {56 vars in 1177 bytes} [Thu May 26 14:56:54 2022] GET /identity/v3/users/209453d7c80c4f44922d6300549194fc => generated 286 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:56:54.616606 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3b5ad9-ea7a-4768-90d4-557c3e10c019 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:56:54.716985 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:56:54.726660 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3b5ad9-ea7a-4768-90d4-557c3e10c019 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:56:54.762492 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a59fb8c-67d0-49f6-bb81-01e8ba6884b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:56:54.763133 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a59fb8c-67d0-49f6-bb81-01e8ba6884b0 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:54.763411 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a59fb8c-67d0-49f6-bb81-01e8ba6884b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:54.763727 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a59fb8c-67d0-49f6-bb81-01e8ba6884b0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:54.834667 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a59fb8c-67d0-49f6-bb81-01e8ba6884b0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:56:54.836090 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a59fb8c-67d0-49f6-bb81-01e8ba6884b0 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:56:54.929352 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:56:54.940000 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 589/1159] 10.222.0.46 () {58 vars in 1282 bytes} [Thu May 26 14:56:54 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 26 14:56:54.946481 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc8bfe8-0569-4054-9628-0ba4c4961912 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:56:55.013796 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:56:55.051126 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc8bfe8-0569-4054-9628-0ba4c4961912 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:56:55.085769 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e450c8a-0bd6-48f2-834c-5a2095469a30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:56:55.086599 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e450c8a-0bd6-48f2-834c-5a2095469a30 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:55.086881 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e450c8a-0bd6-48f2-834c-5a2095469a30 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:55.087155 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e450c8a-0bd6-48f2-834c-5a2095469a30 None admin] PATH_INFO: `/v3/projects/3fa285bf3cf347b1bface45a1758ad7f` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:55.088329 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e450c8a-0bd6-48f2-834c-5a2095469a30 None admin] RBAC: Authorizing `identity:get_project(project_id=3fa285bf3cf347b1bface45a1758ad7f)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:56:55.089934 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e450c8a-0bd6-48f2-834c-5a2095469a30 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:56:55.091683 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 571/1160] 10.222.0.46 () {56 vars in 1186 bytes} [Thu May 26 14:56:54 2022] GET /identity/v3/projects/3fa285bf3cf347b1bface45a1758ad7f => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:56:55.098193 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a59fb8c-67d0-49f6-bb81-01e8ba6884b0 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:56:55.201983 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a59fb8c-67d0-49f6-bb81-01e8ba6884b0 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:56:55.239278 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa30056d-67c4-418a-b2ce-0c6c14caa5fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:56:55.239928 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa30056d-67c4-418a-b2ce-0c6c14caa5fe None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:55.240212 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa30056d-67c4-418a-b2ce-0c6c14caa5fe None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:55.240514 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa30056d-67c4-418a-b2ce-0c6c14caa5fe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:55.241335 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa30056d-67c4-418a-b2ce-0c6c14caa5fe None admin] RBAC: Adding query filter params (role.id=a2e551773cbd4df084020676865b8222, scope.project.id=3fa285bf3cf347b1bface45a1758ad7f, user.id=209453d7c80c4f44922d6300549194fc) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:56:55.241679 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa30056d-67c4-418a-b2ce-0c6c14caa5fe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:56:55.243107 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa30056d-67c4-418a-b2ce-0c6c14caa5fe None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:56:55.248789 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 590/1161] 10.222.0.46 () {56 vars in 1374 bytes} [Thu May 26 14:56:55 2022] GET /identity/v3/role_assignments?user.id=209453d7c80c4f44922d6300549194fc&scope.project.id=3fa285bf3cf347b1bface45a1758ad7f&role.id=a2e551773cbd4df084020676865b8222 => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:56:57.540769 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0574a1-d175-48e4-b6c7-68bdfe188827 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:57.541317 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0574a1-d175-48e4-b6c7-68bdfe188827 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:57.541317 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0574a1-d175-48e4-b6c7-68bdfe188827 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:57.542743 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 572/1162] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 14:56:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 14:56:57.548403 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b979dee2-69c5-4483-a791-37cbaf947115 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:57.548888 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b979dee2-69c5-4483-a791-37cbaf947115 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:57.549000 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b979dee2-69c5-4483-a791-37cbaf947115 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:57.565915 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-b979dee2-69c5-4483-a791-37cbaf947115 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:56:57.567134 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-b979dee2-69c5-4483-a791-37cbaf947115 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:56:57.620301 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:56:57.638634 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 591/1163] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 14:56:57 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 14:56:57.661709 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0574a1-d175-48e4-b6c7-68bdfe188827 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:56:57.770876 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0574a1-d175-48e4-b6c7-68bdfe188827 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:56:57.806881 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0aa3c5b-c207-43f1-81b9-bb9dffc9750b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:56:57.807549 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0aa3c5b-c207-43f1-81b9-bb9dffc9750b None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:57.807858 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0aa3c5b-c207-43f1-81b9-bb9dffc9750b None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:57.808137 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0aa3c5b-c207-43f1-81b9-bb9dffc9750b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:57.811045 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:56:57.879287 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0aa3c5b-c207-43f1-81b9-bb9dffc9750b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:56:57.880763 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0aa3c5b-c207-43f1-81b9-bb9dffc9750b None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:56:57.977072 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 573/1164] 10.222.0.46 () {58 vars in 1282 bytes} [Thu May 26 14:56:57 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 26 14:56:57.983113 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b979dee2-69c5-4483-a791-37cbaf947115 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:56:58.079146 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b979dee2-69c5-4483-a791-37cbaf947115 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:56:58.111365 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1efa48-938d-4bc3-8881-e01baf75fff5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:56:58.112165 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1efa48-938d-4bc3-8881-e01baf75fff5 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:58.112450 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1efa48-938d-4bc3-8881-e01baf75fff5 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:58.112732 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1efa48-938d-4bc3-8881-e01baf75fff5 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:58.119860 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1efa48-938d-4bc3-8881-e01baf75fff5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:56:58.121294 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1efa48-938d-4bc3-8881-e01baf75fff5 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:56:58.136187 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-6f1efa48-938d-4bc3-8881-e01baf75fff5 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. May 26 14:56:58.136187 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:56:58.136187 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain May 26 14:56:58.136187 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) May 26 14:56:58.136187 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 14:56:58.136187 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 14:56:58.136187 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 14:56:58.136187 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 14:56:58.136187 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. May 26 14:56:58.136187 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 26 14:56:58.136187 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 14:56:58.136187 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 26 14:56:58.136187 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:56:58.136187 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:56:58.136187 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:56:58.136187 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:56:58.136187 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:56:58.137376 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:56:58.137376 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:56:58.137376 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:56:58.137376 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:56:58.137376 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:56:58.137376 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:56:58.137376 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 14:56:58.137376 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 14:56:58.137376 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 14:56:58.137376 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 14:56:58.137376 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:56:58.137376 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:56:58.137376 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:56:58.137376 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:56:58.137376 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:56:58.137376 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:56:58.138639 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:56:58.138639 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 26 14:56:58.138639 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:56:58.138639 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 26 14:56:58.138639 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:56:58.138639 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:56:58.138639 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:56:58.138639 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 26 14:56:58.138639 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:56:58.138639 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 26 14:56:58.138639 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain May 26 14:56:58.138639 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 14:56:58.138639 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. May 26 14:56:58.138639 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 26 14:56:58.138639 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 592/1165] 10.222.0.46 () {56 vars in 1117 bytes} [Thu May 26 14:56:57 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:56:58.143326 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0aa3c5b-c207-43f1-81b9-bb9dffc9750b None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:56:58.175053 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:56:58.241288 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0aa3c5b-c207-43f1-81b9-bb9dffc9750b None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:56:58.275510 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523b1074-d6da-4186-9b40-b056878cb251 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:56:58.276157 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523b1074-d6da-4186-9b40-b056878cb251 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:56:58.276439 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523b1074-d6da-4186-9b40-b056878cb251 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:56:58.276713 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523b1074-d6da-4186-9b40-b056878cb251 None admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:56:58.277417 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523b1074-d6da-4186-9b40-b056878cb251 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:56:58.277749 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523b1074-d6da-4186-9b40-b056878cb251 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:56:58.279102 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523b1074-d6da-4186-9b40-b056878cb251 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:56:58.286801 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 574/1166] 10.222.0.46 () {56 vars in 1115 bytes} [Thu May 26 14:56:58 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:57:00.468667 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985c9584-4083-4ebd-a903-decedb092251 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:57:00.469208 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985c9584-4083-4ebd-a903-decedb092251 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:57:00.469208 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985c9584-4083-4ebd-a903-decedb092251 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:57:00.470656 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 593/1167] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 14:57:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 14:57:00.476529 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7432d39a-b582-4dea-9572-e9f885cc2a64 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:57:00.476945 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7432d39a-b582-4dea-9572-e9f885cc2a64 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:57:00.477094 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7432d39a-b582-4dea-9572-e9f885cc2a64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:57:00.493512 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-7432d39a-b582-4dea-9572-e9f885cc2a64 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:57:00.494697 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-7432d39a-b582-4dea-9572-e9f885cc2a64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:57:00.558737 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 575/1168] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 14:57:00 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 14:57:00.581965 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985c9584-4083-4ebd-a903-decedb092251 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:57:00.630576 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:57:00.685920 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985c9584-4083-4ebd-a903-decedb092251 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:57:00.718029 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0918d71-0392-40bc-80d3-fe964b1fe6ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:57:00.718728 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0918d71-0392-40bc-80d3-fe964b1fe6ac None admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:57:00.718961 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0918d71-0392-40bc-80d3-fe964b1fe6ac None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:57:00.719233 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0918d71-0392-40bc-80d3-fe964b1fe6ac None admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:57:00.719962 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0918d71-0392-40bc-80d3-fe964b1fe6ac None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:57:00.721319 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0918d71-0392-40bc-80d3-fe964b1fe6ac None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:57:00.735748 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 594/1169] 10.222.0.46 () {60 vars in 1138 bytes} [Thu May 26 14:57:00 2022] POST /identity/v3/domains => generated 267 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 14:57:02.952019 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a931ac-554e-405b-bbc9-8cc5c31b1480 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:57:02.952521 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a931ac-554e-405b-bbc9-8cc5c31b1480 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:57:02.952521 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a931ac-554e-405b-bbc9-8cc5c31b1480 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:57:02.953883 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 576/1170] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 14:57:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 14:57:02.959107 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412730c3-471d-474b-9ceb-a01fb1298e82 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:57:02.959511 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412730c3-471d-474b-9ceb-a01fb1298e82 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:57:02.959654 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412730c3-471d-474b-9ceb-a01fb1298e82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:57:02.977231 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-412730c3-471d-474b-9ceb-a01fb1298e82 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:57:02.978448 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-412730c3-471d-474b-9ceb-a01fb1298e82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:57:03.043644 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 595/1171] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 14:57:02 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 26 14:57:03.067213 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a931ac-554e-405b-bbc9-8cc5c31b1480 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:57:03.094363 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:57:03.165092 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a931ac-554e-405b-bbc9-8cc5c31b1480 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:57:03.198116 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020e4664-631f-4dab-b222-be2745ca9c7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:57:03.198961 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020e4664-631f-4dab-b222-be2745ca9c7c None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:57:03.199241 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020e4664-631f-4dab-b222-be2745ca9c7c None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:57:03.199513 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020e4664-631f-4dab-b222-be2745ca9c7c None admin] PATH_INFO: `/v3/domains/5053dd8ab8684d528ddeea167d907b4c` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:57:03.210079 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020e4664-631f-4dab-b222-be2745ca9c7c None admin] RBAC: Authorizing `identity:get_domain(domain_id=5053dd8ab8684d528ddeea167d907b4c)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:57:03.211456 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020e4664-631f-4dab-b222-be2745ca9c7c None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:57:03.213069 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 577/1172] 10.222.0.46 () {56 vars in 1183 bytes} [Thu May 26 14:57:03 2022] GET /identity/v3/domains/5053dd8ab8684d528ddeea167d907b4c => generated 267 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:57:03.218857 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412730c3-471d-474b-9ceb-a01fb1298e82 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:57:03.252037 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:57:03.314969 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412730c3-471d-474b-9ceb-a01fb1298e82 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:57:03.347718 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27686309-c7cb-4d80-86fd-8d6da6e47f7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:57:03.348376 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27686309-c7cb-4d80-86fd-8d6da6e47f7d None admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:57:03.348658 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27686309-c7cb-4d80-86fd-8d6da6e47f7d None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:57:03.348935 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27686309-c7cb-4d80-86fd-8d6da6e47f7d None admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:57:03.350373 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27686309-c7cb-4d80-86fd-8d6da6e47f7d None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:57:03.351805 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27686309-c7cb-4d80-86fd-8d6da6e47f7d None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:57:03.376080 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 596/1173] 10.222.0.46 () {60 vars in 1141 bytes} [Thu May 26 14:57:03 2022] POST /identity/v3/projects => generated 361 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 14:57:05.635385 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5daff80d-8a66-4131-b32b-0697c27cf10e None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:57:05.635866 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5daff80d-8a66-4131-b32b-0697c27cf10e None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:57:05.635959 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5daff80d-8a66-4131-b32b-0697c27cf10e None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:57:05.637289 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 578/1174] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 14:57:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 14:57:05.642673 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e505dd-f704-4fbb-aea3-24f10d03daa5 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:57:05.643093 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e505dd-f704-4fbb-aea3-24f10d03daa5 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:57:05.643238 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e505dd-f704-4fbb-aea3-24f10d03daa5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:57:05.659718 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-b5e505dd-f704-4fbb-aea3-24f10d03daa5 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:57:05.660907 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-b5e505dd-f704-4fbb-aea3-24f10d03daa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:57:05.725742 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 597/1175] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 14:57:05 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 26 14:57:05.751184 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5daff80d-8a66-4131-b32b-0697c27cf10e None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:57:05.838636 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:57:05.852397 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5daff80d-8a66-4131-b32b-0697c27cf10e None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:57:05.887327 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648b0e20-11f4-4726-a826-62a1cda22a91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:57:05.888120 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648b0e20-11f4-4726-a826-62a1cda22a91 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:57:05.888399 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648b0e20-11f4-4726-a826-62a1cda22a91 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:57:05.888672 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648b0e20-11f4-4726-a826-62a1cda22a91 None admin] PATH_INFO: `/v3/domains/5053dd8ab8684d528ddeea167d907b4c` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:57:05.889851 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648b0e20-11f4-4726-a826-62a1cda22a91 None admin] RBAC: Authorizing `identity:get_domain(domain_id=5053dd8ab8684d528ddeea167d907b4c)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:57:05.891275 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648b0e20-11f4-4726-a826-62a1cda22a91 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:57:05.892887 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 579/1176] 10.222.0.46 () {56 vars in 1183 bytes} [Thu May 26 14:57:05 2022] GET /identity/v3/domains/5053dd8ab8684d528ddeea167d907b4c => generated 267 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:57:05.898632 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e505dd-f704-4fbb-aea3-24f10d03daa5 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:57:05.917046 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:57:05.997039 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e505dd-f704-4fbb-aea3-24f10d03daa5 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:57:06.031139 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13b91e09-7762-43ae-864d-84aee26b179c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:57:06.031804 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b91e09-7762-43ae-864d-84aee26b179c None admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:57:06.032087 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b91e09-7762-43ae-864d-84aee26b179c None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:57:06.032363 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13b91e09-7762-43ae-864d-84aee26b179c None admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:57:06.033075 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b91e09-7762-43ae-864d-84aee26b179c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:57:06.034541 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13b91e09-7762-43ae-864d-84aee26b179c None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:57:06.053738 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 598/1177] 10.222.0.46 () {60 vars in 1132 bytes} [Thu May 26 14:57:05 2022] POST /identity/v3/users => generated 311 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 14:57:08.290230 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1899b0e6-68dd-40f6-a3a7-17d5a093658c None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:57:08.290790 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1899b0e6-68dd-40f6-a3a7-17d5a093658c None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:57:08.290790 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1899b0e6-68dd-40f6-a3a7-17d5a093658c None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:57:08.292132 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 580/1178] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 14:57:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 14:57:08.297432 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cecaeb7-b043-4be4-abff-197817b7f9aa None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:57:08.297881 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cecaeb7-b043-4be4-abff-197817b7f9aa None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:57:08.298019 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cecaeb7-b043-4be4-abff-197817b7f9aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:57:08.314226 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-5cecaeb7-b043-4be4-abff-197817b7f9aa None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:57:08.315399 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-5cecaeb7-b043-4be4-abff-197817b7f9aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:57:08.380796 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 599/1179] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 14:57:08 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 14:57:08.404004 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1899b0e6-68dd-40f6-a3a7-17d5a093658c None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:57:08.501634 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1899b0e6-68dd-40f6-a3a7-17d5a093658c None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:57:08.534269 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76b25ee-977a-40d0-8fa8-989fcf6a4a26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:57:08.535055 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76b25ee-977a-40d0-8fa8-989fcf6a4a26 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:57:08.535340 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76b25ee-977a-40d0-8fa8-989fcf6a4a26 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:57:08.535617 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76b25ee-977a-40d0-8fa8-989fcf6a4a26 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:57:08.545669 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76b25ee-977a-40d0-8fa8-989fcf6a4a26 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 14:57:08.553983 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-e76b25ee-977a-40d0-8fa8-989fcf6a4a26 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 14:57:08.553983 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:57:08.553983 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:57:08.553983 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:57:08.553983 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:57:08.553983 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:57:08.553983 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:57:08.553983 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:57:08.553983 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:57:08.553983 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:57:08.553983 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:57:08.553983 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:57:08.553983 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 14:57:08.553983 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 14:57:08.553983 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 14:57:08.553983 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 14:57:08.555092 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 14:57:08.555092 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 14:57:08.555092 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 14:57:08.555092 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 14:57:08.555092 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:57:08.555092 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:57:08.555092 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:57:08.555092 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:57:08.555092 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:57:08.555092 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:57:08.555092 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:57:08.555092 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 26 14:57:08.555092 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:57:08.555092 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 26 14:57:08.555092 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:57:08.555092 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:57:08.556323 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:57:08.556323 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 26 14:57:08.556323 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:57:08.556323 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 26 14:57:08.556323 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 14:57:08.556323 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 14:57:08.556323 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 14:57:08.556323 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 14:57:08.556323 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 14:57:08.556323 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 14:57:08.556323 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 14:57:08.556323 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 26 14:57:08.556323 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 581/1180] 10.222.0.46 () {56 vars in 1096 bytes} [Thu May 26 14:57:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:57:08.561937 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cecaeb7-b043-4be4-abff-197817b7f9aa None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:57:08.566522 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:57:08.706714 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cecaeb7-b043-4be4-abff-197817b7f9aa None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:57:08.756142 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2319c6-f77b-4a9b-911e-19e78d70c292 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:57:08.757150 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2319c6-f77b-4a9b-911e-19e78d70c292 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:57:08.757628 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2319c6-f77b-4a9b-911e-19e78d70c292 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:57:08.758068 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2319c6-f77b-4a9b-911e-19e78d70c292 None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:57:08.759253 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2319c6-f77b-4a9b-911e-19e78d70c292 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:57:08.759739 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2319c6-f77b-4a9b-911e-19e78d70c292 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:57:08.761963 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2319c6-f77b-4a9b-911e-19e78d70c292 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:57:08.774017 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 600/1181] 10.222.0.46 () {56 vars in 1099 bytes} [Thu May 26 14:57:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:57:08.779841 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76b25ee-977a-40d0-8fa8-989fcf6a4a26 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:57:08.797807 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:57:08.877569 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76b25ee-977a-40d0-8fa8-989fcf6a4a26 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:57:08.909232 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9487797e-c8e6-46ec-9cf6-978d977dc19e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:57:08.910118 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9487797e-c8e6-46ec-9cf6-978d977dc19e None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:57:08.910399 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9487797e-c8e6-46ec-9cf6-978d977dc19e None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:57:08.910672 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9487797e-c8e6-46ec-9cf6-978d977dc19e None admin] PATH_INFO: `/v3/users/8881340a9ddb49efb68c417d6340a20d` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:57:08.923769 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9487797e-c8e6-46ec-9cf6-978d977dc19e None admin] RBAC: Authorizing `identity:get_user(user_id=8881340a9ddb49efb68c417d6340a20d)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:57:08.925203 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9487797e-c8e6-46ec-9cf6-978d977dc19e None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:57:08.926852 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 582/1182] 10.222.0.46 () {56 vars in 1177 bytes} [Thu May 26 14:57:08 2022] GET /identity/v3/users/8881340a9ddb49efb68c417d6340a20d => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:57:08.933251 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2319c6-f77b-4a9b-911e-19e78d70c292 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:57:08.996170 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:57:09.029848 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2319c6-f77b-4a9b-911e-19e78d70c292 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:57:09.063277 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f5276b-d048-40dd-9c73-64dadd213378 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:57:09.063430 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f5276b-d048-40dd-9c73-64dadd213378 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:57:09.063694 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f5276b-d048-40dd-9c73-64dadd213378 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:57:09.063972 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f5276b-d048-40dd-9c73-64dadd213378 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:57:09.129538 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f5276b-d048-40dd-9c73-64dadd213378 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:57:09.131022 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f5276b-d048-40dd-9c73-64dadd213378 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:57:09.192162 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:57:09.226878 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 601/1183] 10.222.0.46 () {58 vars in 1282 bytes} [Thu May 26 14:57:08 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 26 14:57:09.232918 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9487797e-c8e6-46ec-9cf6-978d977dc19e None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:57:09.304690 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:57:09.328753 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9487797e-c8e6-46ec-9cf6-978d977dc19e None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:57:09.362170 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b35e7c-cda2-408a-a921-2da98619905c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:57:09.363027 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b35e7c-cda2-408a-a921-2da98619905c None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:57:09.363312 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b35e7c-cda2-408a-a921-2da98619905c None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:57:09.363588 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b35e7c-cda2-408a-a921-2da98619905c None admin] PATH_INFO: `/v3/projects/be7dcddd317d4ffb86ec1d537445cb0e` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:57:09.364777 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b35e7c-cda2-408a-a921-2da98619905c None admin] RBAC: Authorizing `identity:get_project(project_id=be7dcddd317d4ffb86ec1d537445cb0e)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:57:09.366271 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b35e7c-cda2-408a-a921-2da98619905c None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:57:09.367977 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 583/1184] 10.222.0.46 () {56 vars in 1186 bytes} [Thu May 26 14:57:09 2022] GET /identity/v3/projects/be7dcddd317d4ffb86ec1d537445cb0e => generated 361 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:57:09.374188 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f5276b-d048-40dd-9c73-64dadd213378 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:57:09.469351 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f5276b-d048-40dd-9c73-64dadd213378 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:57:09.501863 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887ab350-3bbf-4981-ac9b-bce1ff4bb2ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:57:09.502503 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887ab350-3bbf-4981-ac9b-bce1ff4bb2ae None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:57:09.502785 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887ab350-3bbf-4981-ac9b-bce1ff4bb2ae None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:57:09.503312 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887ab350-3bbf-4981-ac9b-bce1ff4bb2ae None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:57:09.504128 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887ab350-3bbf-4981-ac9b-bce1ff4bb2ae None admin] RBAC: Adding query filter params (role.id=a2e551773cbd4df084020676865b8222, scope.project.id=be7dcddd317d4ffb86ec1d537445cb0e, user.id=8881340a9ddb49efb68c417d6340a20d) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:57:09.504442 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887ab350-3bbf-4981-ac9b-bce1ff4bb2ae None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:57:09.505911 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887ab350-3bbf-4981-ac9b-bce1ff4bb2ae None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:57:09.511150 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 602/1185] 10.222.0.46 () {56 vars in 1374 bytes} [Thu May 26 14:57:09 2022] GET /identity/v3/role_assignments?user.id=8881340a9ddb49efb68c417d6340a20d&scope.project.id=be7dcddd317d4ffb86ec1d537445cb0e&role.id=a2e551773cbd4df084020676865b8222 => generated 259 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:57:11.689983 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564b7a60-62bb-4b86-878a-0111c63f840a None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:57:11.690539 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564b7a60-62bb-4b86-878a-0111c63f840a None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:57:11.690539 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564b7a60-62bb-4b86-878a-0111c63f840a None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:57:11.691911 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 584/1186] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 14:57:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 14:57:11.697470 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fcfec1-12b7-4056-8406-007771f34224 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:57:11.697894 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fcfec1-12b7-4056-8406-007771f34224 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:57:11.698028 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fcfec1-12b7-4056-8406-007771f34224 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:57:11.715456 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-b3fcfec1-12b7-4056-8406-007771f34224 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:57:11.716834 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-b3fcfec1-12b7-4056-8406-007771f34224 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:57:11.741064 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:57:11.784911 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 603/1187] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 14:57:11 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 14:57:11.807769 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564b7a60-62bb-4b86-878a-0111c63f840a None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:57:11.910706 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564b7a60-62bb-4b86-878a-0111c63f840a None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:57:11.945205 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e338d96a-b39a-4e32-a395-e1158b1ba76a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:57:11.946030 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e338d96a-b39a-4e32-a395-e1158b1ba76a None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:57:11.946316 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e338d96a-b39a-4e32-a395-e1158b1ba76a None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:57:11.946592 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e338d96a-b39a-4e32-a395-e1158b1ba76a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:57:11.957479 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e338d96a-b39a-4e32-a395-e1158b1ba76a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 14:57:11.965788 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-e338d96a-b39a-4e32-a395-e1158b1ba76a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 14:57:11.965788 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:57:11.965788 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:57:11.965788 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:57:11.965788 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:57:11.965788 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:57:11.965788 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:57:11.965788 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:57:11.965788 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:57:11.965788 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:57:11.965788 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:57:11.965788 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:57:11.965788 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 14:57:11.965788 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 14:57:11.965788 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 14:57:11.965788 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 14:57:11.966925 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 14:57:11.966925 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 14:57:11.966925 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 14:57:11.966925 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 14:57:11.966925 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:57:11.966925 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:57:11.966925 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:57:11.966925 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:57:11.966925 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:57:11.966925 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:57:11.966925 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:57:11.966925 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 26 14:57:11.966925 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:57:11.966925 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 26 14:57:11.966925 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:57:11.966925 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:57:11.967829 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:57:11.967829 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 26 14:57:11.967829 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:57:11.967829 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 26 14:57:11.967829 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 14:57:11.967829 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 14:57:11.967829 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 14:57:11.967829 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 14:57:11.967829 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 14:57:11.967829 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 14:57:11.967829 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 14:57:11.967829 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 26 14:57:11.967829 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 585/1188] 10.222.0.46 () {56 vars in 1096 bytes} [Thu May 26 14:57:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:57:11.973029 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fcfec1-12b7-4056-8406-007771f34224 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:57:12.074861 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fcfec1-12b7-4056-8406-007771f34224 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:57:12.110476 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdf1f2b-db88-477d-8234-a804c61c3208 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:57:12.111154 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdf1f2b-db88-477d-8234-a804c61c3208 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:57:12.111436 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdf1f2b-db88-477d-8234-a804c61c3208 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:57:12.111705 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdf1f2b-db88-477d-8234-a804c61c3208 None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:57:12.112455 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccdf1f2b-db88-477d-8234-a804c61c3208 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:57:12.112764 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccdf1f2b-db88-477d-8234-a804c61c3208 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:57:12.114153 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccdf1f2b-db88-477d-8234-a804c61c3208 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:57:12.122408 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 604/1189] 10.222.0.46 () {56 vars in 1099 bytes} [Thu May 26 14:57:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:57:12.128584 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e338d96a-b39a-4e32-a395-e1158b1ba76a None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:57:12.132617 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:57:12.228345 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e338d96a-b39a-4e32-a395-e1158b1ba76a None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:57:12.261439 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5c8f80-daa7-465f-8e7c-a67ab7e0dccc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:57:12.262329 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5c8f80-daa7-465f-8e7c-a67ab7e0dccc None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:57:12.262615 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5c8f80-daa7-465f-8e7c-a67ab7e0dccc None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:57:12.262898 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5c8f80-daa7-465f-8e7c-a67ab7e0dccc None admin] PATH_INFO: `/v3/users/8881340a9ddb49efb68c417d6340a20d` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:57:12.264076 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5c8f80-daa7-465f-8e7c-a67ab7e0dccc None admin] RBAC: Authorizing `identity:get_user(user_id=8881340a9ddb49efb68c417d6340a20d)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:57:12.265552 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5c8f80-daa7-465f-8e7c-a67ab7e0dccc None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:57:12.267187 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 586/1190] 10.222.0.46 () {56 vars in 1177 bytes} [Thu May 26 14:57:12 2022] GET /identity/v3/users/8881340a9ddb49efb68c417d6340a20d => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:57:12.273115 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdf1f2b-db88-477d-8234-a804c61c3208 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:57:12.297342 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:57:12.370692 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdf1f2b-db88-477d-8234-a804c61c3208 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:57:12.404030 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e385a570-9741-4e55-aa87-5ca764f274c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:57:12.404881 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e385a570-9741-4e55-aa87-5ca764f274c0 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:57:12.405156 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e385a570-9741-4e55-aa87-5ca764f274c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:57:12.405430 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e385a570-9741-4e55-aa87-5ca764f274c0 None admin] PATH_INFO: `/v3/projects/be7dcddd317d4ffb86ec1d537445cb0e` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:57:12.406609 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e385a570-9741-4e55-aa87-5ca764f274c0 None admin] RBAC: Authorizing `identity:get_project(project_id=be7dcddd317d4ffb86ec1d537445cb0e)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:57:12.408026 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e385a570-9741-4e55-aa87-5ca764f274c0 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:57:12.409736 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 605/1191] 10.222.0.46 () {56 vars in 1186 bytes} [Thu May 26 14:57:12 2022] GET /identity/v3/projects/be7dcddd317d4ffb86ec1d537445cb0e => generated 361 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:57:12.415620 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5c8f80-daa7-465f-8e7c-a67ab7e0dccc None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:57:12.484128 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:57:12.515549 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5c8f80-daa7-465f-8e7c-a67ab7e0dccc None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:57:12.549426 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24126dc-5d52-425b-aad9-14f28417e345 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:57:12.550502 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24126dc-5d52-425b-aad9-14f28417e345 None admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:57:12.550787 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24126dc-5d52-425b-aad9-14f28417e345 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:57:12.551067 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24126dc-5d52-425b-aad9-14f28417e345 None admin] PATH_INFO: `/v3/projects/be7dcddd317d4ffb86ec1d537445cb0e/users/8881340a9ddb49efb68c417d6340a20d/roles/a2e551773cbd4df084020676865b8222` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:57:12.553608 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24126dc-5d52-425b-aad9-14f28417e345 None admin] RBAC: Authorizing `identity:create_grant(project_id=be7dcddd317d4ffb86ec1d537445cb0e, user_id=8881340a9ddb49efb68c417d6340a20d, role_id=a2e551773cbd4df084020676865b8222)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:57:12.555063 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a24126dc-5d52-425b-aad9-14f28417e345 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:57:12.568236 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 587/1192] 10.222.0.46 () {58 vars in 1439 bytes} [Thu May 26 14:57:12 2022] PUT /identity/v3/projects/be7dcddd317d4ffb86ec1d537445cb0e/users/8881340a9ddb49efb68c417d6340a20d/roles/a2e551773cbd4df084020676865b8222 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 14:57:14.712609 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c138f0-4e12-4131-8fe3-8d5f4d97d9a2 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:57:14.713187 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c138f0-4e12-4131-8fe3-8d5f4d97d9a2 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:57:14.713187 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c138f0-4e12-4131-8fe3-8d5f4d97d9a2 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:57:14.714710 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 606/1193] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 14:57:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 14:57:14.719870 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a086fc0-2e0c-4730-b279-0ceb3cbb1ac5 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:57:14.720304 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a086fc0-2e0c-4730-b279-0ceb3cbb1ac5 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:57:14.720412 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a086fc0-2e0c-4730-b279-0ceb3cbb1ac5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:57:14.739624 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-7a086fc0-2e0c-4730-b279-0ceb3cbb1ac5 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:57:14.740837 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-7a086fc0-2e0c-4730-b279-0ceb3cbb1ac5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:57:14.808444 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 588/1194] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 14:57:14 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 14:57:14.831887 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c138f0-4e12-4131-8fe3-8d5f4d97d9a2 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:57:14.921670 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:57:14.932617 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c138f0-4e12-4131-8fe3-8d5f4d97d9a2 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:57:14.966082 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc4f72f-0537-4214-9e98-6fca391f6975 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:57:14.966871 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc4f72f-0537-4214-9e98-6fca391f6975 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:57:14.967158 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc4f72f-0537-4214-9e98-6fca391f6975 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:57:14.967436 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc4f72f-0537-4214-9e98-6fca391f6975 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:57:14.977774 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc4f72f-0537-4214-9e98-6fca391f6975 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 14:57:14.985340 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-0bc4f72f-0537-4214-9e98-6fca391f6975 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 14:57:14.985340 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:57:14.985340 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:57:14.985340 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:57:14.985340 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:57:14.985340 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:57:14.985340 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:57:14.985340 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:57:14.985340 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:57:14.985340 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:57:14.985340 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:57:14.985340 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:57:14.985340 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 14:57:14.985340 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 14:57:14.985340 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 14:57:14.985340 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 14:57:14.986515 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 14:57:14.986515 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 14:57:14.986515 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 14:57:14.986515 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 14:57:14.986515 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:57:14.986515 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:57:14.986515 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:57:14.986515 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:57:14.986515 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:57:14.986515 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:57:14.986515 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:57:14.986515 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 26 14:57:14.986515 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:57:14.986515 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 26 14:57:14.986515 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:57:14.986515 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:57:14.987527 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:57:14.987527 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 26 14:57:14.987527 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:57:14.987527 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 26 14:57:14.987527 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 14:57:14.987527 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 14:57:14.987527 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 14:57:14.987527 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 14:57:14.987527 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 14:57:14.987527 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 14:57:14.987527 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 14:57:14.987527 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 26 14:57:14.987527 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 607/1195] 10.222.0.46 () {56 vars in 1096 bytes} [Thu May 26 14:57:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:57:14.992688 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a086fc0-2e0c-4730-b279-0ceb3cbb1ac5 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:57:15.050618 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:57:15.091082 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a086fc0-2e0c-4730-b279-0ceb3cbb1ac5 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:57:15.123773 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3448dd02-2145-47aa-b7bb-2c24785d3f63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:57:15.124438 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3448dd02-2145-47aa-b7bb-2c24785d3f63 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:57:15.124722 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3448dd02-2145-47aa-b7bb-2c24785d3f63 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:57:15.125001 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3448dd02-2145-47aa-b7bb-2c24785d3f63 None admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:57:15.125778 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3448dd02-2145-47aa-b7bb-2c24785d3f63 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:57:15.126092 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3448dd02-2145-47aa-b7bb-2c24785d3f63 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:57:15.127462 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3448dd02-2145-47aa-b7bb-2c24785d3f63 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:57:15.136299 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 589/1196] 10.222.0.46 () {56 vars in 1099 bytes} [Thu May 26 14:57:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:57:15.142253 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc4f72f-0537-4214-9e98-6fca391f6975 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:57:15.238883 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc4f72f-0537-4214-9e98-6fca391f6975 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:57:15.272476 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fe2ef5-a62e-404d-8384-335aa21f7b15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:57:15.273339 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fe2ef5-a62e-404d-8384-335aa21f7b15 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:57:15.273654 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fe2ef5-a62e-404d-8384-335aa21f7b15 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:57:15.273929 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fe2ef5-a62e-404d-8384-335aa21f7b15 None admin] PATH_INFO: `/v3/users/8881340a9ddb49efb68c417d6340a20d` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:57:15.275095 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fe2ef5-a62e-404d-8384-335aa21f7b15 None admin] RBAC: Authorizing `identity:get_user(user_id=8881340a9ddb49efb68c417d6340a20d)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:57:15.276525 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fe2ef5-a62e-404d-8384-335aa21f7b15 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:57:15.278182 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 608/1197] 10.222.0.46 () {56 vars in 1177 bytes} [Thu May 26 14:57:15 2022] GET /identity/v3/users/8881340a9ddb49efb68c417d6340a20d => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:57:15.283957 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3448dd02-2145-47aa-b7bb-2c24785d3f63 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:57:15.381724 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3448dd02-2145-47aa-b7bb-2c24785d3f63 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:57:15.403337 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:57:15.417705 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccaab589-f738-47bf-8f38-8c02617c3f26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:57:15.418355 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccaab589-f738-47bf-8f38-8c02617c3f26 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:57:15.418630 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccaab589-f738-47bf-8f38-8c02617c3f26 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:57:15.418898 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccaab589-f738-47bf-8f38-8c02617c3f26 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:57:15.486580 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccaab589-f738-47bf-8f38-8c02617c3f26 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:57:15.488026 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccaab589-f738-47bf-8f38-8c02617c3f26 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:57:15.583326 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 590/1198] 10.222.0.46 () {58 vars in 1282 bytes} [Thu May 26 14:57:15 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 26 14:57:15.589162 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fe2ef5-a62e-404d-8384-335aa21f7b15 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:57:15.608281 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:57:15.689476 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fe2ef5-a62e-404d-8384-335aa21f7b15 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:57:15.722471 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94bde252-fdb8-4a2f-85e2-79868b130c49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:57:15.723323 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bde252-fdb8-4a2f-85e2-79868b130c49 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:57:15.723611 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bde252-fdb8-4a2f-85e2-79868b130c49 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:57:15.724415 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94bde252-fdb8-4a2f-85e2-79868b130c49 None admin] PATH_INFO: `/v3/projects/be7dcddd317d4ffb86ec1d537445cb0e` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:57:15.725645 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bde252-fdb8-4a2f-85e2-79868b130c49 None admin] RBAC: Authorizing `identity:get_project(project_id=be7dcddd317d4ffb86ec1d537445cb0e)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:57:15.727076 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94bde252-fdb8-4a2f-85e2-79868b130c49 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:57:15.728772 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 609/1199] 10.222.0.46 () {56 vars in 1186 bytes} [Thu May 26 14:57:15 2022] GET /identity/v3/projects/be7dcddd317d4ffb86ec1d537445cb0e => generated 361 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:57:15.734900 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccaab589-f738-47bf-8f38-8c02617c3f26 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:57:15.758751 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:57:15.832867 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccaab589-f738-47bf-8f38-8c02617c3f26 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:57:15.866306 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d2d858-ad2b-4aa5-bc44-b757fa43f6c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:57:15.866887 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d2d858-ad2b-4aa5-bc44-b757fa43f6c7 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:57:15.867170 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d2d858-ad2b-4aa5-bc44-b757fa43f6c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:57:15.867447 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d2d858-ad2b-4aa5-bc44-b757fa43f6c7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:57:15.868263 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d2d858-ad2b-4aa5-bc44-b757fa43f6c7 None admin] RBAC: Adding query filter params (role.id=a2e551773cbd4df084020676865b8222, scope.project.id=be7dcddd317d4ffb86ec1d537445cb0e, user.id=8881340a9ddb49efb68c417d6340a20d) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:57:15.868576 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d2d858-ad2b-4aa5-bc44-b757fa43f6c7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:57:15.870039 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d2d858-ad2b-4aa5-bc44-b757fa43f6c7 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:57:15.875694 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 591/1200] 10.222.0.46 () {56 vars in 1374 bytes} [Thu May 26 14:57:15 2022] GET /identity/v3/role_assignments?user.id=8881340a9ddb49efb68c417d6340a20d&scope.project.id=be7dcddd317d4ffb86ec1d537445cb0e&role.id=a2e551773cbd4df084020676865b8222 => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:57:25.718502 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=57 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:57:25.873081 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=57 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:58:16.768128 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee6f471-a557-4ad3-9a40-c2ce72cb52df None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:58:16.768742 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee6f471-a557-4ad3-9a40-c2ce72cb52df None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:58:16.768742 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee6f471-a557-4ad3-9a40-c2ce72cb52df None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:58:16.770386 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 610/1201] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 14:58:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 14:58:16.776024 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e689f60b-0850-421d-82fb-439d8783fbd3 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:58:16.776505 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e689f60b-0850-421d-82fb-439d8783fbd3 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:58:16.776607 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e689f60b-0850-421d-82fb-439d8783fbd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:58:16.798698 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-e689f60b-0850-421d-82fb-439d8783fbd3 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:58:16.799988 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-e689f60b-0850-421d-82fb-439d8783fbd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:58:16.882756 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 592/1202] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 14:58:16 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 14:58:16.910198 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee6f471-a557-4ad3-9a40-c2ce72cb52df None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:58:17.046055 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee6f471-a557-4ad3-9a40-c2ce72cb52df None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:58:17.089420 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16092b1-3a65-4ae0-b157-f4ce91aad96f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:58:17.090384 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16092b1-3a65-4ae0-b157-f4ce91aad96f None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:58:17.090683 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16092b1-3a65-4ae0-b157-f4ce91aad96f None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:58:17.090973 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16092b1-3a65-4ae0-b157-f4ce91aad96f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:58:17.103527 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16092b1-3a65-4ae0-b157-f4ce91aad96f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:58:17.105005 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16092b1-3a65-4ae0-b157-f4ce91aad96f None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:58:17.128040 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-f16092b1-3a65-4ae0-b157-f4ce91aad96f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 14:58:17.128040 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:58:17.128040 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:58:17.128040 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:58:17.128040 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:58:17.128040 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:58:17.128040 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:58:17.128040 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:58:17.128040 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:58:17.128040 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:58:17.128040 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:58:17.128040 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:58:17.128040 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 14:58:17.128040 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 14:58:17.128040 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 14:58:17.129111 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 14:58:17.129111 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:58:17.129111 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:58:17.129111 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:58:17.129111 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:58:17.129111 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:58:17.129111 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:58:17.129111 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:58:17.129111 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 26 14:58:17.129111 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:58:17.129111 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 26 14:58:17.129111 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:58:17.129111 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:58:17.129111 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:58:17.129111 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 26 14:58:17.129111 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:58:17.129111 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 26 14:58:17.135951 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 14:58:17.135951 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 14:58:17.135951 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 14:58:17.135951 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 26 14:58:17.135951 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 611/1203] 10.222.0.46 () {56 vars in 1108 bytes} [Thu May 26 14:58:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 14:58:17.138015 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e689f60b-0850-421d-82fb-439d8783fbd3 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:58:17.274636 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e689f60b-0850-421d-82fb-439d8783fbd3 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:58:17.314993 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:58:17.324496 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca13fedc-cb4a-49ca-b831-9994d6be2b13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:58:17.325191 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca13fedc-cb4a-49ca-b831-9994d6be2b13 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:58:17.325521 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca13fedc-cb4a-49ca-b831-9994d6be2b13 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:58:17.325829 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca13fedc-cb4a-49ca-b831-9994d6be2b13 None admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:58:17.326556 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca13fedc-cb4a-49ca-b831-9994d6be2b13 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:58:17.326873 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca13fedc-cb4a-49ca-b831-9994d6be2b13 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:58:17.328298 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca13fedc-cb4a-49ca-b831-9994d6be2b13 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:58:17.349027 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 593/1204] 10.222.0.46 () {56 vars in 1109 bytes} [Thu May 26 14:58:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:58:17.358577 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16092b1-3a65-4ae0-b157-f4ce91aad96f None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:58:17.429636 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:58:17.484070 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16092b1-3a65-4ae0-b157-f4ce91aad96f None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:58:17.535879 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fa8497-f62e-42fc-9fd9-20762046ef4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:58:17.536600 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fa8497-f62e-42fc-9fd9-20762046ef4c None admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:58:17.536920 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fa8497-f62e-42fc-9fd9-20762046ef4c None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:58:17.537198 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fa8497-f62e-42fc-9fd9-20762046ef4c None admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:58:17.538036 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fa8497-f62e-42fc-9fd9-20762046ef4c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:58:17.539608 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fa8497-f62e-42fc-9fd9-20762046ef4c None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:58:17.569940 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 612/1205] 10.222.0.46 () {60 vars in 1132 bytes} [Thu May 26 14:58:17 2022] POST /identity/v3/users => generated 251 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 14:58:20.119098 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd614509-d76b-4e54-b9e5-c3a4766191ba None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:58:20.119661 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd614509-d76b-4e54-b9e5-c3a4766191ba None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:58:20.119661 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd614509-d76b-4e54-b9e5-c3a4766191ba None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:58:20.121035 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 594/1206] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 14:58:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 14:58:20.126986 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac58db5-4200-44ad-8d68-aacd0ad2c959 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:58:20.127447 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac58db5-4200-44ad-8d68-aacd0ad2c959 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:58:20.127537 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac58db5-4200-44ad-8d68-aacd0ad2c959 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:58:20.146794 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-4ac58db5-4200-44ad-8d68-aacd0ad2c959 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:58:20.148071 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-4ac58db5-4200-44ad-8d68-aacd0ad2c959 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:58:20.229376 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 613/1207] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 14:58:20 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 14:58:20.254695 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd614509-d76b-4e54-b9e5-c3a4766191ba None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:58:20.373809 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd614509-d76b-4e54-b9e5-c3a4766191ba None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:58:20.418752 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c5928e-d838-4c8e-bb24-1a8fe71da5e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:58:20.419560 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c5928e-d838-4c8e-bb24-1a8fe71da5e1 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:58:20.419903 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c5928e-d838-4c8e-bb24-1a8fe71da5e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:58:20.420233 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c5928e-d838-4c8e-bb24-1a8fe71da5e1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:58:20.435989 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c5928e-d838-4c8e-bb24-1a8fe71da5e1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 14:58:20.445633 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-29c5928e-d838-4c8e-bb24-1a8fe71da5e1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 26 14:58:20.445633 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:58:20.445633 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:58:20.445633 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:58:20.445633 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:58:20.445633 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:58:20.445633 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:58:20.445633 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:58:20.445633 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:58:20.445633 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:58:20.445633 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:58:20.445633 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:58:20.445633 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 14:58:20.445633 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 14:58:20.445633 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 14:58:20.445633 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 14:58:20.446680 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 14:58:20.446680 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 14:58:20.446680 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 14:58:20.446680 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 14:58:20.446680 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:58:20.446680 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:58:20.446680 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:58:20.446680 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:58:20.446680 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:58:20.446680 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:58:20.446680 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:58:20.446680 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 26 14:58:20.446680 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:58:20.446680 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 26 14:58:20.446680 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:58:20.446680 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:58:20.448412 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:58:20.448412 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 26 14:58:20.448412 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:58:20.448412 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 26 14:58:20.448412 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 14:58:20.448412 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 14:58:20.448412 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 14:58:20.448412 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 14:58:20.448412 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 14:58:20.448412 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 14:58:20.448412 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 26 14:58:20.448412 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 26 14:58:20.448412 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 595/1208] 10.222.0.46 () {56 vars in 1102 bytes} [Thu May 26 14:58:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:58:20.455266 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac58db5-4200-44ad-8d68-aacd0ad2c959 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:58:20.504018 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:58:20.573257 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac58db5-4200-44ad-8d68-aacd0ad2c959 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:58:20.623486 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd96200-2c83-4098-9bf5-e3d4ff6eb90c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:58:20.624274 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd96200-2c83-4098-9bf5-e3d4ff6eb90c None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:58:20.624544 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd96200-2c83-4098-9bf5-e3d4ff6eb90c None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:58:20.624831 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd96200-2c83-4098-9bf5-e3d4ff6eb90c None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:58:20.625752 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd96200-2c83-4098-9bf5-e3d4ff6eb90c None admin] RBAC: Adding query filter params (name=service) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:58:20.626067 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd96200-2c83-4098-9bf5-e3d4ff6eb90c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:58:20.627937 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbd96200-2c83-4098-9bf5-e3d4ff6eb90c None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:58:20.644089 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 614/1209] 10.222.0.46 () {56 vars in 1103 bytes} [Thu May 26 14:58:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:58:20.649535 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c5928e-d838-4c8e-bb24-1a8fe71da5e1 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:58:20.712394 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:58:20.780208 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c5928e-d838-4c8e-bb24-1a8fe71da5e1 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:58:20.817290 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5812562f-fd8f-4b9b-8500-2cf2f53fb846 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:58:20.818135 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5812562f-fd8f-4b9b-8500-2cf2f53fb846 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:58:20.818416 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5812562f-fd8f-4b9b-8500-2cf2f53fb846 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:58:20.818704 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5812562f-fd8f-4b9b-8500-2cf2f53fb846 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:58:20.830328 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5812562f-fd8f-4b9b-8500-2cf2f53fb846 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:58:20.831803 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5812562f-fd8f-4b9b-8500-2cf2f53fb846 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:58:20.851625 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-5812562f-fd8f-4b9b-8500-2cf2f53fb846 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 14:58:20.851625 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:58:20.851625 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:58:20.851625 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:58:20.851625 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:58:20.851625 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:58:20.851625 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:58:20.851625 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:58:20.851625 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:58:20.851625 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:58:20.851625 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:58:20.851625 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:58:20.851625 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 14:58:20.851625 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 14:58:20.851625 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 14:58:20.852717 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 14:58:20.852717 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:58:20.852717 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:58:20.852717 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:58:20.852717 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:58:20.852717 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:58:20.852717 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:58:20.852717 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:58:20.852717 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 26 14:58:20.852717 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:58:20.852717 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 26 14:58:20.852717 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:58:20.852717 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:58:20.852717 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:58:20.852717 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 26 14:58:20.852717 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:58:20.852717 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 26 14:58:20.853644 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 14:58:20.853644 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 14:58:20.853644 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 14:58:20.853644 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 26 14:58:20.860674 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 596/1210] 10.222.0.46 () {56 vars in 1108 bytes} [Thu May 26 14:58:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:58:20.861844 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd96200-2c83-4098-9bf5-e3d4ff6eb90c None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:58:20.988678 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:58:20.991797 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbd96200-2c83-4098-9bf5-e3d4ff6eb90c None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:58:21.032181 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d2edf6-15bd-4522-886e-dee3cc4f69ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:58:21.032851 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d2edf6-15bd-4522-886e-dee3cc4f69ce None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:58:21.033139 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d2edf6-15bd-4522-886e-dee3cc4f69ce None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:58:21.033423 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d2edf6-15bd-4522-886e-dee3cc4f69ce None admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:58:21.034259 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d2edf6-15bd-4522-886e-dee3cc4f69ce None admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:58:21.034577 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d2edf6-15bd-4522-886e-dee3cc4f69ce None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:58:21.036030 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d2edf6-15bd-4522-886e-dee3cc4f69ce None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:58:21.058292 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 615/1211] 10.222.0.46 () {56 vars in 1109 bytes} [Thu May 26 14:58:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:58:21.059491 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5812562f-fd8f-4b9b-8500-2cf2f53fb846 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:58:21.182152 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5812562f-fd8f-4b9b-8500-2cf2f53fb846 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:58:21.183884 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:58:21.222955 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b383bb8-5841-4fae-b3a2-dca5bdf952ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:58:21.223771 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b383bb8-5841-4fae-b3a2-dca5bdf952ce None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:58:21.224051 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b383bb8-5841-4fae-b3a2-dca5bdf952ce None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:58:21.224333 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b383bb8-5841-4fae-b3a2-dca5bdf952ce None admin] PATH_INFO: `/v3/users/placement` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:58:21.236901 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b383bb8-5841-4fae-b3a2-dca5bdf952ce None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:58:21.238758 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b383bb8-5841-4fae-b3a2-dca5bdf952ce None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:58:21.256609 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-6b383bb8-5841-4fae-b3a2-dca5bdf952ce None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 26 14:58:21.256609 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:58:21.256609 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:58:21.256609 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:58:21.256609 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:58:21.256609 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:58:21.256609 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:58:21.256609 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:58:21.256609 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:58:21.256609 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:58:21.256609 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:58:21.256609 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:58:21.256609 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 14:58:21.256609 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 14:58:21.256609 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 14:58:21.256609 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 14:58:21.257816 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:58:21.257816 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:58:21.257816 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 14:58:21.257816 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 14:58:21.257816 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 14:58:21.257816 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 14:58:21.257816 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:58:21.257816 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:58:21.257816 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:58:21.257816 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:58:21.257816 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:58:21.257816 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 26 14:58:21.257816 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:58:21.257816 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 26 14:58:21.257816 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:58:21.257816 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:58:21.257816 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:58:21.257816 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 26 14:58:21.263528 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:58:21.263528 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 26 14:58:21.263528 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 14:58:21.263528 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 14:58:21.263528 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 14:58:21.263528 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 14:58:21.263528 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 14:58:21.263528 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 14:58:21.263528 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 26 14:58:21.263528 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 26 14:58:21.263528 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 597/1212] 10.222.0.46 () {56 vars in 1108 bytes} [Thu May 26 14:58:21 2022] GET /identity/v3/users/placement => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 14:58:21.264259 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d2edf6-15bd-4522-886e-dee3cc4f69ce None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:58:21.390491 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d2edf6-15bd-4522-886e-dee3cc4f69ce None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:58:21.433865 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe33068-daf2-4cfc-b09a-332cfe5c9c8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:58:21.434844 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe33068-daf2-4cfc-b09a-332cfe5c9c8d None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:58:21.435139 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe33068-daf2-4cfc-b09a-332cfe5c9c8d None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:58:21.435421 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe33068-daf2-4cfc-b09a-332cfe5c9c8d None admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:58:21.436269 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe33068-daf2-4cfc-b09a-332cfe5c9c8d None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:58:21.436583 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe33068-daf2-4cfc-b09a-332cfe5c9c8d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:58:21.438109 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe33068-daf2-4cfc-b09a-332cfe5c9c8d None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:58:21.458204 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 616/1213] 10.222.0.46 () {56 vars in 1143 bytes} [Thu May 26 14:58:21 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:58:21.464159 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b383bb8-5841-4fae-b3a2-dca5bdf952ce None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:58:21.582755 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b383bb8-5841-4fae-b3a2-dca5bdf952ce None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:58:21.627814 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a041e09d-a499-44ab-a220-8c2968e931ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:58:21.628687 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a041e09d-a499-44ab-a220-8c2968e931ed None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:58:21.629128 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a041e09d-a499-44ab-a220-8c2968e931ed None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:58:21.629633 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a041e09d-a499-44ab-a220-8c2968e931ed None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:58:21.663704 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:58:21.706085 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a041e09d-a499-44ab-a220-8c2968e931ed None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:58:21.707551 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a041e09d-a499-44ab-a220-8c2968e931ed None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:58:21.827676 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 598/1214] 10.222.0.46 () {58 vars in 1282 bytes} [Thu May 26 14:58:21 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 26 14:58:21.835394 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe33068-daf2-4cfc-b09a-332cfe5c9c8d None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:58:21.858479 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:58:21.956138 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe33068-daf2-4cfc-b09a-332cfe5c9c8d None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:58:22.000645 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1939c4be-a215-4d76-bbd3-b8a9e0d7f193 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:58:22.002514 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1939c4be-a215-4d76-bbd3-b8a9e0d7f193 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:58:22.002770 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1939c4be-a215-4d76-bbd3-b8a9e0d7f193 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:58:22.003053 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1939c4be-a215-4d76-bbd3-b8a9e0d7f193 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:58:22.018514 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1939c4be-a215-4d76-bbd3-b8a9e0d7f193 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:58:22.020050 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1939c4be-a215-4d76-bbd3-b8a9e0d7f193 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:58:22.038834 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-1939c4be-a215-4d76-bbd3-b8a9e0d7f193 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 14:58:22.038834 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:58:22.038834 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:58:22.038834 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:58:22.038834 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:58:22.038834 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:58:22.038834 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:58:22.038834 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:58:22.038834 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:58:22.038834 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:58:22.038834 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:58:22.038834 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:58:22.038834 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 14:58:22.038834 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 14:58:22.038834 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 14:58:22.040264 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 14:58:22.040264 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:58:22.040264 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:58:22.040264 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:58:22.040264 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:58:22.040264 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:58:22.040264 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:58:22.040264 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:58:22.040264 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 26 14:58:22.040264 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:58:22.040264 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 26 14:58:22.040264 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:58:22.040264 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:58:22.040264 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:58:22.040264 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 26 14:58:22.040264 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:58:22.040264 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 26 14:58:22.041292 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 14:58:22.041292 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 14:58:22.041292 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 14:58:22.041292 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 26 14:58:22.041292 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 617/1215] 10.222.0.46 () {56 vars in 1108 bytes} [Thu May 26 14:58:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:58:22.046338 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a041e09d-a499-44ab-a220-8c2968e931ed None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:58:22.122853 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:58:22.174668 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a041e09d-a499-44ab-a220-8c2968e931ed None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:58:22.216585 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5982ae5f-ec3a-4c4b-b80a-100be1c6add7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:58:22.217246 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5982ae5f-ec3a-4c4b-b80a-100be1c6add7 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:58:22.217577 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5982ae5f-ec3a-4c4b-b80a-100be1c6add7 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:58:22.217870 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5982ae5f-ec3a-4c4b-b80a-100be1c6add7 None admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:58:22.218647 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5982ae5f-ec3a-4c4b-b80a-100be1c6add7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:58:22.218952 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5982ae5f-ec3a-4c4b-b80a-100be1c6add7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:58:22.220432 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5982ae5f-ec3a-4c4b-b80a-100be1c6add7 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:58:22.237526 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 599/1216] 10.222.0.46 () {56 vars in 1109 bytes} [Thu May 26 14:58:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 14:58:22.239315 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1939c4be-a215-4d76-bbd3-b8a9e0d7f193 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:58:22.320837 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:58:22.357399 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1939c4be-a215-4d76-bbd3-b8a9e0d7f193 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:58:22.402960 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2310efad-1389-48b5-a727-acc501cd9a0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:58:22.403827 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2310efad-1389-48b5-a727-acc501cd9a0f None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:58:22.404121 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2310efad-1389-48b5-a727-acc501cd9a0f None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:58:22.404464 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2310efad-1389-48b5-a727-acc501cd9a0f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:58:22.413425 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2310efad-1389-48b5-a727-acc501cd9a0f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:58:22.414916 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2310efad-1389-48b5-a727-acc501cd9a0f None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:58:22.434855 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-2310efad-1389-48b5-a727-acc501cd9a0f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 14:58:22.434855 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:58:22.434855 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:58:22.434855 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:58:22.434855 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:58:22.434855 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:58:22.434855 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:58:22.434855 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:58:22.434855 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:58:22.434855 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:58:22.434855 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:58:22.434855 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:58:22.434855 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 14:58:22.434855 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 14:58:22.434855 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 14:58:22.436009 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 14:58:22.436009 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:58:22.436009 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:58:22.436009 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:58:22.436009 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:58:22.436009 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:58:22.436009 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:58:22.436009 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:58:22.436009 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 26 14:58:22.436009 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:58:22.436009 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 26 14:58:22.436009 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:58:22.436009 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:58:22.436009 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:58:22.436009 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 26 14:58:22.436009 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:58:22.436009 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 26 14:58:22.437199 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 14:58:22.437199 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 14:58:22.437199 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 14:58:22.437199 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 14:58:22.437199 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 14:58:22.437199 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 14:58:22.437199 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 14:58:22.437199 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 26 14:58:22.441306 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 618/1217] 10.222.0.46 () {56 vars in 1111 bytes} [Thu May 26 14:58:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:58:22.442232 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5982ae5f-ec3a-4c4b-b80a-100be1c6add7 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:58:22.555301 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5982ae5f-ec3a-4c4b-b80a-100be1c6add7 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:58:22.578158 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:58:22.591607 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85a744f-ef89-4fa0-b083-642a21cfcb0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:58:22.592391 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85a744f-ef89-4fa0-b083-642a21cfcb0e None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:58:22.592774 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85a744f-ef89-4fa0-b083-642a21cfcb0e None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:58:22.593146 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85a744f-ef89-4fa0-b083-642a21cfcb0e None admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:58:22.594069 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85a744f-ef89-4fa0-b083-642a21cfcb0e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:58:22.594477 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85a744f-ef89-4fa0-b083-642a21cfcb0e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:58:22.596544 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85a744f-ef89-4fa0-b083-642a21cfcb0e None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:58:22.614714 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 600/1218] 10.222.0.46 () {56 vars in 1148 bytes} [Thu May 26 14:58:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:58:22.617950 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2310efad-1389-48b5-a727-acc501cd9a0f None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:58:22.732016 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2310efad-1389-48b5-a727-acc501cd9a0f None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:58:22.766957 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:58:22.773742 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdc15ef-e69a-4fd9-a271-1d49f2fe380f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:58:22.774394 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdc15ef-e69a-4fd9-a271-1d49f2fe380f None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:58:22.774681 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdc15ef-e69a-4fd9-a271-1d49f2fe380f None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:58:22.774955 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdc15ef-e69a-4fd9-a271-1d49f2fe380f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:58:22.775793 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdc15ef-e69a-4fd9-a271-1d49f2fe380f None admin] RBAC: Adding query filter params (role.id=eb76213820fa4c4e9b0a5d5ac2a08b41, scope.project.id=a17fc8227d4c4da6b2847b350e912ee3, user.id=bf55a00e3b084aa3b22c25a90e6a773c) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:58:22.776106 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdc15ef-e69a-4fd9-a271-1d49f2fe380f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:58:22.777568 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdc15ef-e69a-4fd9-a271-1d49f2fe380f None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:58:22.783893 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 619/1219] 10.222.0.46 () {56 vars in 1374 bytes} [Thu May 26 14:58:22 2022] GET /identity/v3/role_assignments?user.id=bf55a00e3b084aa3b22c25a90e6a773c&scope.project.id=a17fc8227d4c4da6b2847b350e912ee3&role.id=eb76213820fa4c4e9b0a5d5ac2a08b41 => generated 259 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:58:25.238357 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e288acea-f32a-4848-9e02-6cdc8dde55dc None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:58:25.238942 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e288acea-f32a-4848-9e02-6cdc8dde55dc None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:58:25.238942 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e288acea-f32a-4848-9e02-6cdc8dde55dc None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:58:25.240276 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 601/1220] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 14:58:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 14:58:25.246930 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223f59ce-b8c4-4f7b-98ea-f1134af88eac None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:58:25.247405 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223f59ce-b8c4-4f7b-98ea-f1134af88eac None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:58:25.247505 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-223f59ce-b8c4-4f7b-98ea-f1134af88eac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:58:25.267461 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-223f59ce-b8c4-4f7b-98ea-f1134af88eac None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:58:25.268739 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-223f59ce-b8c4-4f7b-98ea-f1134af88eac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:58:25.347583 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 620/1221] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 14:58:25 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 26 14:58:25.373702 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e288acea-f32a-4848-9e02-6cdc8dde55dc None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:58:25.510644 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e288acea-f32a-4848-9e02-6cdc8dde55dc None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:58:25.559087 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4466f3aa-a427-438b-82f7-f64b0dbdd2a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:58:25.560070 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4466f3aa-a427-438b-82f7-f64b0dbdd2a1 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:58:25.560357 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4466f3aa-a427-438b-82f7-f64b0dbdd2a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:58:25.560703 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4466f3aa-a427-438b-82f7-f64b0dbdd2a1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:58:25.575570 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4466f3aa-a427-438b-82f7-f64b0dbdd2a1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 14:58:25.584847 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-4466f3aa-a427-438b-82f7-f64b0dbdd2a1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 26 14:58:25.584847 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:58:25.584847 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:58:25.584847 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:58:25.584847 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:58:25.584847 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:58:25.584847 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:58:25.584847 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:58:25.584847 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:58:25.584847 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:58:25.584847 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:58:25.584847 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:58:25.584847 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 14:58:25.584847 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 14:58:25.584847 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 14:58:25.584847 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 14:58:25.586063 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 14:58:25.586063 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 14:58:25.586063 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 14:58:25.586063 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 14:58:25.586063 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:58:25.586063 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:58:25.586063 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:58:25.586063 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:58:25.586063 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:58:25.586063 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:58:25.586063 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:58:25.586063 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 26 14:58:25.586063 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:58:25.586063 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 26 14:58:25.586063 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:58:25.586063 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:58:25.587239 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:58:25.587239 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 26 14:58:25.587239 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:58:25.587239 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 26 14:58:25.587239 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 14:58:25.587239 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 14:58:25.587239 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 14:58:25.587239 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 14:58:25.587239 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 14:58:25.587239 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 14:58:25.587239 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 26 14:58:25.587239 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 26 14:58:25.587239 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 602/1222] 10.222.0.46 () {56 vars in 1102 bytes} [Thu May 26 14:58:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:58:25.592555 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223f59ce-b8c4-4f7b-98ea-f1134af88eac None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:58:25.711443 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-223f59ce-b8c4-4f7b-98ea-f1134af88eac None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:58:25.735765 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:58:25.747847 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c0fc65-914a-4b5f-ac9d-827f35c37043 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:58:25.748575 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c0fc65-914a-4b5f-ac9d-827f35c37043 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:58:25.748896 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c0fc65-914a-4b5f-ac9d-827f35c37043 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:58:25.749203 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c0fc65-914a-4b5f-ac9d-827f35c37043 None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:58:25.750009 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c0fc65-914a-4b5f-ac9d-827f35c37043 None admin] RBAC: Adding query filter params (name=service) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:58:25.750378 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c0fc65-914a-4b5f-ac9d-827f35c37043 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:58:25.752234 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c0fc65-914a-4b5f-ac9d-827f35c37043 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:58:25.761200 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 621/1223] 10.222.0.46 () {56 vars in 1103 bytes} [Thu May 26 14:58:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 14:58:25.768001 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4466f3aa-a427-438b-82f7-f64b0dbdd2a1 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:58:25.813590 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:58:25.873416 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4466f3aa-a427-438b-82f7-f64b0dbdd2a1 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:58:25.907030 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e326a2c-f1ec-45f7-8c2b-5bcc623bfa4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:58:25.908397 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e326a2c-f1ec-45f7-8c2b-5bcc623bfa4a None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:58:25.908749 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e326a2c-f1ec-45f7-8c2b-5bcc623bfa4a None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:58:25.909034 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e326a2c-f1ec-45f7-8c2b-5bcc623bfa4a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:58:25.920405 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e326a2c-f1ec-45f7-8c2b-5bcc623bfa4a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:58:25.921876 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e326a2c-f1ec-45f7-8c2b-5bcc623bfa4a None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:58:25.939868 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-4e326a2c-f1ec-45f7-8c2b-5bcc623bfa4a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 14:58:25.939868 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:58:25.939868 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:58:25.939868 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:58:25.939868 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:58:25.939868 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:58:25.939868 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:58:25.939868 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:58:25.939868 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:58:25.939868 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:58:25.939868 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:58:25.939868 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:58:25.939868 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 14:58:25.939868 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 14:58:25.939868 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 14:58:25.940810 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 14:58:25.940810 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:58:25.940810 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:58:25.940810 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:58:25.940810 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:58:25.940810 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:58:25.940810 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:58:25.940810 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:58:25.940810 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 26 14:58:25.940810 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:58:25.940810 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 26 14:58:25.940810 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:58:25.940810 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:58:25.940810 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:58:25.940810 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 26 14:58:25.940810 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:58:25.940810 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 26 14:58:25.942093 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 14:58:25.942093 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 14:58:25.942093 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 14:58:25.942093 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 26 14:58:25.942093 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 603/1224] 10.222.0.46 () {56 vars in 1108 bytes} [Thu May 26 14:58:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:58:25.947443 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c0fc65-914a-4b5f-ac9d-827f35c37043 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:58:26.051316 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c0fc65-914a-4b5f-ac9d-827f35c37043 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:58:26.088207 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67934d0c-78a3-4da2-af83-766cf7d05362 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:58:26.088872 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67934d0c-78a3-4da2-af83-766cf7d05362 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:58:26.089158 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67934d0c-78a3-4da2-af83-766cf7d05362 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:58:26.089440 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67934d0c-78a3-4da2-af83-766cf7d05362 None admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:58:26.090186 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67934d0c-78a3-4da2-af83-766cf7d05362 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:58:26.090491 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67934d0c-78a3-4da2-af83-766cf7d05362 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:58:26.091930 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67934d0c-78a3-4da2-af83-766cf7d05362 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:58:26.104660 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 622/1225] 10.222.0.46 () {56 vars in 1109 bytes} [Thu May 26 14:58:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 14:58:26.110964 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e326a2c-f1ec-45f7-8c2b-5bcc623bfa4a None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:58:26.210405 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:58:26.217395 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e326a2c-f1ec-45f7-8c2b-5bcc623bfa4a None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:58:26.254319 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e7b20f-155a-4989-8bc9-4e6a67550f4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:58:26.255285 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e7b20f-155a-4989-8bc9-4e6a67550f4c None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:58:26.255641 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e7b20f-155a-4989-8bc9-4e6a67550f4c None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:58:26.255932 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e7b20f-155a-4989-8bc9-4e6a67550f4c None admin] PATH_INFO: `/v3/users/placement` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:58:26.263629 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e7b20f-155a-4989-8bc9-4e6a67550f4c None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:58:26.265072 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e7b20f-155a-4989-8bc9-4e6a67550f4c None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:58:26.279982 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-a7e7b20f-155a-4989-8bc9-4e6a67550f4c None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 26 14:58:26.279982 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:58:26.279982 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:58:26.279982 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:58:26.279982 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:58:26.279982 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:58:26.279982 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:58:26.279982 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:58:26.279982 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:58:26.279982 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:58:26.279982 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:58:26.279982 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:58:26.279982 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 14:58:26.279982 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 14:58:26.279982 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 14:58:26.279982 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 14:58:26.281122 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:58:26.281122 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:58:26.281122 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 14:58:26.281122 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 14:58:26.281122 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 14:58:26.281122 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 14:58:26.281122 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:58:26.281122 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:58:26.281122 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:58:26.281122 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:58:26.281122 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:58:26.281122 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 26 14:58:26.281122 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:58:26.281122 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 26 14:58:26.281122 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:58:26.281122 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:58:26.281122 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:58:26.281122 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 26 14:58:26.286885 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:58:26.286885 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 26 14:58:26.286885 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 14:58:26.286885 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 14:58:26.286885 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 14:58:26.286885 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 14:58:26.286885 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 14:58:26.286885 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 14:58:26.286885 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 26 14:58:26.286885 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 26 14:58:26.286885 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 604/1226] 10.222.0.46 () {56 vars in 1108 bytes} [Thu May 26 14:58:26 2022] GET /identity/v3/users/placement => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 26 14:58:26.287558 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67934d0c-78a3-4da2-af83-766cf7d05362 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:58:26.338950 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:58:26.395835 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67934d0c-78a3-4da2-af83-766cf7d05362 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:58:26.434046 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac4e99c-041b-427c-9bfd-2a062ce2bbaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:58:26.434718 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac4e99c-041b-427c-9bfd-2a062ce2bbaf None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:58:26.434999 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac4e99c-041b-427c-9bfd-2a062ce2bbaf None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:58:26.435272 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac4e99c-041b-427c-9bfd-2a062ce2bbaf None admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:58:26.436095 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac4e99c-041b-427c-9bfd-2a062ce2bbaf None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:58:26.436428 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac4e99c-041b-427c-9bfd-2a062ce2bbaf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:58:26.437871 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac4e99c-041b-427c-9bfd-2a062ce2bbaf None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:58:26.455657 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 623/1227] 10.222.0.46 () {56 vars in 1143 bytes} [Thu May 26 14:58:26 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:58:26.461466 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e7b20f-155a-4989-8bc9-4e6a67550f4c None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:58:26.565237 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e7b20f-155a-4989-8bc9-4e6a67550f4c None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:58:26.604973 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e062428-9956-4285-994c-4bf045f304ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:58:26.605779 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e062428-9956-4285-994c-4bf045f304ca None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:58:26.606059 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e062428-9956-4285-994c-4bf045f304ca None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:58:26.606325 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e062428-9956-4285-994c-4bf045f304ca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:58:26.618371 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e062428-9956-4285-994c-4bf045f304ca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:58:26.619933 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e062428-9956-4285-994c-4bf045f304ca None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:58:26.639691 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-3e062428-9956-4285-994c-4bf045f304ca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 14:58:26.639691 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:58:26.639691 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:58:26.639691 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:58:26.639691 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:58:26.639691 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:58:26.639691 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:58:26.639691 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:58:26.639691 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:58:26.639691 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:58:26.639691 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:58:26.639691 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:58:26.639691 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 14:58:26.639691 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 14:58:26.639691 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 14:58:26.640822 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 14:58:26.640822 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:58:26.640822 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:58:26.640822 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:58:26.640822 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:58:26.640822 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:58:26.640822 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:58:26.640822 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:58:26.640822 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 26 14:58:26.640822 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:58:26.640822 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 26 14:58:26.640822 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:58:26.640822 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:58:26.640822 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:58:26.640822 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 26 14:58:26.640822 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:58:26.640822 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 26 14:58:26.642714 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 14:58:26.642714 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 14:58:26.642714 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 14:58:26.642714 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 26 14:58:26.642714 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 605/1228] 10.222.0.46 () {56 vars in 1108 bytes} [Thu May 26 14:58:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:58:26.647337 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac4e99c-041b-427c-9bfd-2a062ce2bbaf None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:58:26.757265 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac4e99c-041b-427c-9bfd-2a062ce2bbaf None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:58:26.759620 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:58:26.795165 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b7e51e-091e-413c-8ca4-d1a95f500e58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:58:26.795879 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b7e51e-091e-413c-8ca4-d1a95f500e58 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:58:26.796179 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b7e51e-091e-413c-8ca4-d1a95f500e58 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:58:26.796481 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b7e51e-091e-413c-8ca4-d1a95f500e58 None admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:58:26.797207 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b7e51e-091e-413c-8ca4-d1a95f500e58 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:58:26.797662 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b7e51e-091e-413c-8ca4-d1a95f500e58 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:58:26.799625 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b7e51e-091e-413c-8ca4-d1a95f500e58 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:58:26.812903 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 624/1229] 10.222.0.46 () {56 vars in 1109 bytes} [Thu May 26 14:58:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:58:26.819571 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e062428-9956-4285-994c-4bf045f304ca None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:58:26.830318 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:58:26.930477 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e062428-9956-4285-994c-4bf045f304ca None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:58:26.967176 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a665ea-5749-41f6-9172-dfcf7e23496d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:58:26.967976 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a665ea-5749-41f6-9172-dfcf7e23496d None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:58:26.968261 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a665ea-5749-41f6-9172-dfcf7e23496d None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:58:26.968557 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a665ea-5749-41f6-9172-dfcf7e23496d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:58:26.976512 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a665ea-5749-41f6-9172-dfcf7e23496d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:58:26.978049 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a665ea-5749-41f6-9172-dfcf7e23496d None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:58:26.993384 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-d4a665ea-5749-41f6-9172-dfcf7e23496d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 14:58:26.993384 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:58:26.993384 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:58:26.993384 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:58:26.993384 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:58:26.993384 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:58:26.993384 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:58:26.993384 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:58:26.993384 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:58:26.993384 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:58:26.993384 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:58:26.993384 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:58:26.993384 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 14:58:26.993384 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 14:58:26.993384 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 14:58:26.994675 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 14:58:26.994675 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:58:26.994675 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:58:26.994675 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:58:26.994675 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:58:26.994675 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:58:26.994675 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:58:26.994675 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:58:26.994675 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 26 14:58:26.994675 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:58:26.994675 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 26 14:58:26.994675 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:58:26.994675 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:58:26.994675 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:58:26.994675 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 26 14:58:26.994675 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:58:26.994675 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 26 14:58:26.995579 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 14:58:26.995579 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 14:58:26.995579 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 14:58:26.995579 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 14:58:26.995579 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 14:58:26.995579 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 14:58:26.995579 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 14:58:26.995579 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 26 14:58:26.995579 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 606/1230] 10.222.0.46 () {56 vars in 1111 bytes} [Thu May 26 14:58:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:58:27.000946 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b7e51e-091e-413c-8ca4-d1a95f500e58 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:58:27.111380 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b7e51e-091e-413c-8ca4-d1a95f500e58 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:58:27.152957 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9fbca4-4e85-47e1-acaf-51e2a792dfc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:58:27.153976 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9fbca4-4e85-47e1-acaf-51e2a792dfc1 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:58:27.154359 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9fbca4-4e85-47e1-acaf-51e2a792dfc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:58:27.154716 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9fbca4-4e85-47e1-acaf-51e2a792dfc1 None admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:58:27.155810 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9fbca4-4e85-47e1-acaf-51e2a792dfc1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:58:27.156277 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9fbca4-4e85-47e1-acaf-51e2a792dfc1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:58:27.158169 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9fbca4-4e85-47e1-acaf-51e2a792dfc1 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:58:27.172794 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 625/1231] 10.222.0.46 () {56 vars in 1148 bytes} [Thu May 26 14:58:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:58:27.179419 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a665ea-5749-41f6-9172-dfcf7e23496d None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:58:27.260452 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:58:27.299216 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a665ea-5749-41f6-9172-dfcf7e23496d None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:58:27.336620 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fec3ab-5b7a-4a5a-8202-27e3417221e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:58:27.337570 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fec3ab-5b7a-4a5a-8202-27e3417221e6 None admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:58:27.337926 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fec3ab-5b7a-4a5a-8202-27e3417221e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:58:27.338169 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fec3ab-5b7a-4a5a-8202-27e3417221e6 None admin] PATH_INFO: `/v3/projects/a17fc8227d4c4da6b2847b350e912ee3/users/bf55a00e3b084aa3b22c25a90e6a773c/roles/eb76213820fa4c4e9b0a5d5ac2a08b41` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:58:27.354102 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95fec3ab-5b7a-4a5a-8202-27e3417221e6 None admin] RBAC: Authorizing `identity:create_grant(project_id=a17fc8227d4c4da6b2847b350e912ee3, user_id=bf55a00e3b084aa3b22c25a90e6a773c, role_id=eb76213820fa4c4e9b0a5d5ac2a08b41)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:58:27.355640 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95fec3ab-5b7a-4a5a-8202-27e3417221e6 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:58:27.380615 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 607/1232] 10.222.0.46 () {58 vars in 1439 bytes} [Thu May 26 14:58:27 2022] PUT /identity/v3/projects/a17fc8227d4c4da6b2847b350e912ee3/users/bf55a00e3b084aa3b22c25a90e6a773c/roles/eb76213820fa4c4e9b0a5d5ac2a08b41 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 14:58:29.806642 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9eaf14-c159-4dcb-8ce1-89393b716df4 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:58:29.807188 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9eaf14-c159-4dcb-8ce1-89393b716df4 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:58:29.807188 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9eaf14-c159-4dcb-8ce1-89393b716df4 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:58:29.808698 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 626/1233] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 14:58:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 14:58:29.814497 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1e3e96-9eab-4fe9-a4d1-615a7ecc5634 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:58:29.814924 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1e3e96-9eab-4fe9-a4d1-615a7ecc5634 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:58:29.815129 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1e3e96-9eab-4fe9-a4d1-615a7ecc5634 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:58:29.831926 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-1d1e3e96-9eab-4fe9-a4d1-615a7ecc5634 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:58:29.833122 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-1d1e3e96-9eab-4fe9-a4d1-615a7ecc5634 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:58:29.902045 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 608/1234] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 14:58:29 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 26 14:58:29.925923 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9eaf14-c159-4dcb-8ce1-89393b716df4 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:58:29.956258 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:58:30.038293 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9eaf14-c159-4dcb-8ce1-89393b716df4 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:58:30.075762 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960ecbdf-163c-4928-97a0-d96adf06fc2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:58:30.076649 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960ecbdf-163c-4928-97a0-d96adf06fc2e None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:58:30.076954 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960ecbdf-163c-4928-97a0-d96adf06fc2e None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:58:30.077272 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960ecbdf-163c-4928-97a0-d96adf06fc2e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:58:30.087389 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960ecbdf-163c-4928-97a0-d96adf06fc2e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 14:58:30.096447 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-960ecbdf-163c-4928-97a0-d96adf06fc2e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 26 14:58:30.096447 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:58:30.096447 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:58:30.096447 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:58:30.096447 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:58:30.096447 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:58:30.096447 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:58:30.096447 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:58:30.096447 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:58:30.096447 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:58:30.096447 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:58:30.096447 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:58:30.096447 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 14:58:30.096447 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 14:58:30.096447 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 14:58:30.096447 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 14:58:30.097443 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 14:58:30.097443 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 14:58:30.097443 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 14:58:30.097443 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 14:58:30.097443 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:58:30.097443 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:58:30.097443 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:58:30.097443 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:58:30.097443 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:58:30.097443 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:58:30.097443 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:58:30.097443 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 26 14:58:30.097443 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:58:30.097443 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 26 14:58:30.097443 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:58:30.097443 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:58:30.098345 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:58:30.098345 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 26 14:58:30.098345 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:58:30.098345 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 26 14:58:30.098345 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 14:58:30.098345 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 14:58:30.098345 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 14:58:30.098345 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 14:58:30.098345 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 14:58:30.098345 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 14:58:30.098345 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 26 14:58:30.098345 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 26 14:58:30.098345 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 627/1235] 10.222.0.46 () {56 vars in 1102 bytes} [Thu May 26 14:58:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:58:30.104118 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1e3e96-9eab-4fe9-a4d1-615a7ecc5634 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:58:30.158253 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:58:30.217844 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1e3e96-9eab-4fe9-a4d1-615a7ecc5634 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:58:30.256153 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24dead71-e86b-4250-9bbb-75f2f9da2c37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:58:30.256836 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dead71-e86b-4250-9bbb-75f2f9da2c37 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:58:30.257115 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dead71-e86b-4250-9bbb-75f2f9da2c37 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:58:30.257388 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dead71-e86b-4250-9bbb-75f2f9da2c37 None admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:58:30.258159 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24dead71-e86b-4250-9bbb-75f2f9da2c37 None admin] RBAC: Adding query filter params (name=service) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:58:30.258464 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24dead71-e86b-4250-9bbb-75f2f9da2c37 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:58:30.259838 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24dead71-e86b-4250-9bbb-75f2f9da2c37 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:58:30.268905 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 609/1236] 10.222.0.46 () {56 vars in 1103 bytes} [Thu May 26 14:58:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:58:30.274661 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960ecbdf-163c-4928-97a0-d96adf06fc2e None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:58:30.366057 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:58:30.385254 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960ecbdf-163c-4928-97a0-d96adf06fc2e None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:58:30.428836 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa552a26-4d71-4c92-9db5-8d32c86e3526 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:58:30.429585 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa552a26-4d71-4c92-9db5-8d32c86e3526 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:58:30.429868 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa552a26-4d71-4c92-9db5-8d32c86e3526 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:58:30.430183 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa552a26-4d71-4c92-9db5-8d32c86e3526 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:58:30.441233 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa552a26-4d71-4c92-9db5-8d32c86e3526 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:58:30.442748 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa552a26-4d71-4c92-9db5-8d32c86e3526 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:58:30.462182 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-aa552a26-4d71-4c92-9db5-8d32c86e3526 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 14:58:30.462182 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:58:30.462182 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:58:30.462182 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:58:30.462182 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:58:30.462182 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:58:30.462182 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:58:30.462182 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:58:30.462182 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:58:30.462182 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:58:30.462182 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:58:30.462182 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:58:30.462182 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 14:58:30.462182 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 14:58:30.462182 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 14:58:30.463282 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 14:58:30.463282 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:58:30.463282 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:58:30.463282 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:58:30.463282 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:58:30.463282 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:58:30.463282 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:58:30.463282 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:58:30.463282 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 26 14:58:30.463282 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:58:30.463282 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 26 14:58:30.463282 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:58:30.463282 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:58:30.463282 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:58:30.463282 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 26 14:58:30.463282 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:58:30.463282 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 26 14:58:30.468927 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 14:58:30.468927 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 14:58:30.468927 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 14:58:30.468927 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 26 14:58:30.468927 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 628/1237] 10.222.0.46 () {56 vars in 1108 bytes} [Thu May 26 14:58:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:58:30.469259 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24dead71-e86b-4250-9bbb-75f2f9da2c37 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:58:30.572835 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24dead71-e86b-4250-9bbb-75f2f9da2c37 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:58:30.582546 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:58:30.611887 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17724bd-fb0a-419c-a04b-041b04b883a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:58:30.612555 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17724bd-fb0a-419c-a04b-041b04b883a3 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:58:30.612847 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17724bd-fb0a-419c-a04b-041b04b883a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:58:30.613122 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17724bd-fb0a-419c-a04b-041b04b883a3 None admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:58:30.613885 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17724bd-fb0a-419c-a04b-041b04b883a3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:58:30.614194 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17724bd-fb0a-419c-a04b-041b04b883a3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:58:30.615566 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17724bd-fb0a-419c-a04b-041b04b883a3 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:58:30.629400 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 610/1238] 10.222.0.46 () {56 vars in 1109 bytes} [Thu May 26 14:58:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:58:30.635217 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa552a26-4d71-4c92-9db5-8d32c86e3526 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:58:30.745959 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa552a26-4d71-4c92-9db5-8d32c86e3526 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:58:30.783548 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c494ec-b7d7-4b0c-9cc7-320a91178222 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:58:30.784395 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c494ec-b7d7-4b0c-9cc7-320a91178222 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:58:30.784682 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c494ec-b7d7-4b0c-9cc7-320a91178222 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:58:30.784986 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c494ec-b7d7-4b0c-9cc7-320a91178222 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:58:30.788021 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:58:30.794191 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c494ec-b7d7-4b0c-9cc7-320a91178222 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:58:30.795863 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c494ec-b7d7-4b0c-9cc7-320a91178222 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:58:30.811044 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-93c494ec-b7d7-4b0c-9cc7-320a91178222 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 26 14:58:30.811044 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:58:30.811044 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:58:30.811044 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:58:30.811044 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:58:30.811044 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:58:30.811044 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:58:30.811044 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:58:30.811044 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:58:30.811044 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:58:30.811044 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:58:30.811044 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:58:30.811044 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 14:58:30.811044 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 14:58:30.811044 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 14:58:30.811044 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 14:58:30.812504 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:58:30.812504 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:58:30.812504 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 14:58:30.812504 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 14:58:30.812504 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 14:58:30.812504 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 14:58:30.812504 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:58:30.812504 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:58:30.812504 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:58:30.812504 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:58:30.812504 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:58:30.812504 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 26 14:58:30.812504 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:58:30.812504 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 26 14:58:30.812504 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:58:30.812504 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:58:30.812504 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:58:30.812504 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 26 14:58:30.814117 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:58:30.814117 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 26 14:58:30.814117 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 14:58:30.814117 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 14:58:30.814117 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 14:58:30.814117 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 14:58:30.814117 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 14:58:30.814117 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 14:58:30.814117 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 26 14:58:30.814117 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 26 14:58:30.814117 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 629/1239] 10.222.0.46 () {56 vars in 1108 bytes} [Thu May 26 14:58:30 2022] GET /identity/v3/users/placement => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:58:30.819023 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17724bd-fb0a-419c-a04b-041b04b883a3 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:58:30.929244 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17724bd-fb0a-419c-a04b-041b04b883a3 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:58:30.964737 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb50389-c991-49eb-9b9e-2b1a6efaa665 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:58:30.965410 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb50389-c991-49eb-9b9e-2b1a6efaa665 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:58:30.965720 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb50389-c991-49eb-9b9e-2b1a6efaa665 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:58:30.965992 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb50389-c991-49eb-9b9e-2b1a6efaa665 None admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:58:30.966817 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb50389-c991-49eb-9b9e-2b1a6efaa665 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:58:30.967121 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb50389-c991-49eb-9b9e-2b1a6efaa665 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:58:30.968551 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb50389-c991-49eb-9b9e-2b1a6efaa665 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:58:30.987221 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 611/1240] 10.222.0.46 () {56 vars in 1143 bytes} [Thu May 26 14:58:30 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 14:58:30.993264 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c494ec-b7d7-4b0c-9cc7-320a91178222 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:58:31.106167 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c494ec-b7d7-4b0c-9cc7-320a91178222 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:58:31.141575 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1ab7b5-c27f-4f66-a5d3-5dc502728e8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:58:31.142253 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1ab7b5-c27f-4f66-a5d3-5dc502728e8a None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:58:31.142545 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1ab7b5-c27f-4f66-a5d3-5dc502728e8a None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:58:31.142826 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1ab7b5-c27f-4f66-a5d3-5dc502728e8a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:58:31.224086 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:58:31.232017 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1ab7b5-c27f-4f66-a5d3-5dc502728e8a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:58:31.233914 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1ab7b5-c27f-4f66-a5d3-5dc502728e8a None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:58:31.335653 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 630/1241] 10.222.0.46 () {58 vars in 1282 bytes} [Thu May 26 14:58:30 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 26 14:58:31.341766 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb50389-c991-49eb-9b9e-2b1a6efaa665 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:58:31.370386 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:58:31.458084 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb50389-c991-49eb-9b9e-2b1a6efaa665 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:58:31.498573 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b740c83e-5012-40f5-ae43-d4e9ad770d1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:58:31.499349 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b740c83e-5012-40f5-ae43-d4e9ad770d1c None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:58:31.499671 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b740c83e-5012-40f5-ae43-d4e9ad770d1c None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:58:31.499921 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b740c83e-5012-40f5-ae43-d4e9ad770d1c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:58:31.510812 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b740c83e-5012-40f5-ae43-d4e9ad770d1c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:58:31.512439 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b740c83e-5012-40f5-ae43-d4e9ad770d1c None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:58:31.530277 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-b740c83e-5012-40f5-ae43-d4e9ad770d1c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 14:58:31.530277 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:58:31.530277 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:58:31.530277 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:58:31.530277 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:58:31.530277 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:58:31.530277 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:58:31.530277 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:58:31.530277 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:58:31.530277 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:58:31.530277 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:58:31.530277 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:58:31.530277 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 14:58:31.530277 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 14:58:31.530277 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 14:58:31.531659 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 14:58:31.531659 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:58:31.531659 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:58:31.531659 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:58:31.531659 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:58:31.531659 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:58:31.531659 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:58:31.531659 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:58:31.531659 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 26 14:58:31.531659 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:58:31.531659 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 26 14:58:31.531659 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:58:31.531659 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:58:31.531659 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:58:31.531659 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 26 14:58:31.531659 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:58:31.531659 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 26 14:58:31.533106 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 14:58:31.533106 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 14:58:31.533106 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 14:58:31.533106 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 26 14:58:31.533106 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 612/1242] 10.222.0.46 () {56 vars in 1108 bytes} [Thu May 26 14:58:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:58:31.538115 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1ab7b5-c27f-4f66-a5d3-5dc502728e8a None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:58:31.647432 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:58:31.654631 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1ab7b5-c27f-4f66-a5d3-5dc502728e8a None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:58:31.693383 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6472e021-e791-4027-a530-6f6cd1511623 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:58:31.694073 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6472e021-e791-4027-a530-6f6cd1511623 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:58:31.694376 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6472e021-e791-4027-a530-6f6cd1511623 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:58:31.694638 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6472e021-e791-4027-a530-6f6cd1511623 None admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:58:31.695358 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6472e021-e791-4027-a530-6f6cd1511623 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:58:31.695671 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6472e021-e791-4027-a530-6f6cd1511623 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:58:31.697022 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6472e021-e791-4027-a530-6f6cd1511623 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:58:31.709154 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 631/1243] 10.222.0.46 () {56 vars in 1109 bytes} [Thu May 26 14:58:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 14:58:31.715491 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b740c83e-5012-40f5-ae43-d4e9ad770d1c None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:58:31.805717 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:58:31.835241 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b740c83e-5012-40f5-ae43-d4e9ad770d1c None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:58:31.871445 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4735870f-2a6a-4291-b0f4-3293389a7e24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:58:31.872322 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4735870f-2a6a-4291-b0f4-3293389a7e24 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:58:31.872604 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4735870f-2a6a-4291-b0f4-3293389a7e24 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:58:31.872896 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4735870f-2a6a-4291-b0f4-3293389a7e24 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:58:31.880876 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4735870f-2a6a-4291-b0f4-3293389a7e24 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:58:31.882470 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4735870f-2a6a-4291-b0f4-3293389a7e24 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:58:31.897751 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-4735870f-2a6a-4291-b0f4-3293389a7e24 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 14:58:31.897751 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:58:31.897751 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:58:31.897751 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:58:31.897751 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:58:31.897751 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:58:31.897751 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:58:31.897751 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:58:31.897751 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:58:31.897751 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:58:31.897751 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:58:31.897751 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:58:31.897751 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 14:58:31.897751 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 14:58:31.897751 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 14:58:31.898857 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 14:58:31.898857 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:58:31.898857 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:58:31.898857 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:58:31.898857 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:58:31.898857 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:58:31.898857 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:58:31.898857 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:58:31.898857 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 26 14:58:31.898857 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:58:31.898857 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 26 14:58:31.898857 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:58:31.898857 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:58:31.898857 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:58:31.898857 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 26 14:58:31.898857 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:58:31.898857 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 26 14:58:31.904690 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 14:58:31.904690 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 14:58:31.904690 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 14:58:31.904690 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 14:58:31.904690 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 14:58:31.904690 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 14:58:31.904690 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 14:58:31.904690 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 26 14:58:31.904690 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 613/1244] 10.222.0.46 () {56 vars in 1111 bytes} [Thu May 26 14:58:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:58:31.905295 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6472e021-e791-4027-a530-6f6cd1511623 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:58:32.019745 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6472e021-e791-4027-a530-6f6cd1511623 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:58:32.054564 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfd23d1-db5a-4e9e-9295-9b83ec3e4c4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:58:32.055229 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfd23d1-db5a-4e9e-9295-9b83ec3e4c4b None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:58:32.055572 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfd23d1-db5a-4e9e-9295-9b83ec3e4c4b None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:58:32.055851 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfd23d1-db5a-4e9e-9295-9b83ec3e4c4b None admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:58:32.056634 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfd23d1-db5a-4e9e-9295-9b83ec3e4c4b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:58:32.056943 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfd23d1-db5a-4e9e-9295-9b83ec3e4c4b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:58:32.058391 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfd23d1-db5a-4e9e-9295-9b83ec3e4c4b None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:58:32.070907 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 632/1245] 10.222.0.46 () {56 vars in 1148 bytes} [Thu May 26 14:58:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:58:32.078333 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4735870f-2a6a-4291-b0f4-3293389a7e24 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:58:32.190935 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4735870f-2a6a-4291-b0f4-3293389a7e24 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:58:32.228782 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581cf1be-5f34-4ae7-959e-bfdfed544754 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:58:32.229585 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581cf1be-5f34-4ae7-959e-bfdfed544754 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:58:32.229871 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581cf1be-5f34-4ae7-959e-bfdfed544754 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:58:32.230147 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581cf1be-5f34-4ae7-959e-bfdfed544754 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:58:32.231197 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581cf1be-5f34-4ae7-959e-bfdfed544754 None admin] RBAC: Adding query filter params (role.id=eb76213820fa4c4e9b0a5d5ac2a08b41, scope.project.id=a17fc8227d4c4da6b2847b350e912ee3, user.id=bf55a00e3b084aa3b22c25a90e6a773c) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:58:32.231591 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581cf1be-5f34-4ae7-959e-bfdfed544754 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:58:32.233249 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581cf1be-5f34-4ae7-959e-bfdfed544754 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:58:32.240513 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 614/1246] 10.222.0.46 () {56 vars in 1374 bytes} [Thu May 26 14:58:32 2022] GET /identity/v3/role_assignments?user.id=bf55a00e3b084aa3b22c25a90e6a773c&scope.project.id=a17fc8227d4c4da6b2847b350e912ee3&role.id=eb76213820fa4c4e9b0a5d5ac2a08b41 => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:58:34.671189 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af39def-b8c9-4e75-9206-f0726a3b947b None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:58:34.672007 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af39def-b8c9-4e75-9206-f0726a3b947b None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:58:34.672007 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af39def-b8c9-4e75-9206-f0726a3b947b None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:58:34.673821 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 633/1247] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 14:58:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 14:58:34.679774 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4735f98b-7ca6-4795-8c00-4ad42b3ab678 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:58:34.680483 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4735f98b-7ca6-4795-8c00-4ad42b3ab678 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:58:34.680483 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4735f98b-7ca6-4795-8c00-4ad42b3ab678 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:58:34.686032 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:58:34.700265 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-4735f98b-7ca6-4795-8c00-4ad42b3ab678 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:58:34.701708 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-4735f98b-7ca6-4795-8c00-4ad42b3ab678 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:58:34.786366 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 615/1248] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 14:58:34 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 14:58:34.811630 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af39def-b8c9-4e75-9206-f0726a3b947b None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:58:34.834353 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:58:34.935445 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af39def-b8c9-4e75-9206-f0726a3b947b None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:58:34.977913 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5418f4-482d-4981-9ce7-81d0a0034e5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:58:34.978725 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5418f4-482d-4981-9ce7-81d0a0034e5f None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:58:34.979065 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5418f4-482d-4981-9ce7-81d0a0034e5f None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:58:34.979363 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5418f4-482d-4981-9ce7-81d0a0034e5f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:58:34.991000 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5418f4-482d-4981-9ce7-81d0a0034e5f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 14:58:34.998706 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-7b5418f4-482d-4981-9ce7-81d0a0034e5f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 14:58:34.998706 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:58:34.998706 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:58:34.998706 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:58:34.998706 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:58:34.998706 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:58:34.998706 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:58:34.998706 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:58:34.998706 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:58:34.998706 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:58:34.998706 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:58:34.998706 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:58:34.998706 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 14:58:34.998706 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 14:58:34.998706 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 14:58:34.998706 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 14:58:35.000742 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 14:58:35.000742 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 14:58:35.000742 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 14:58:35.000742 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 14:58:35.000742 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:58:35.000742 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:58:35.000742 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:58:35.000742 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:58:35.000742 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:58:35.000742 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:58:35.000742 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:58:35.000742 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 26 14:58:35.000742 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:58:35.000742 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 26 14:58:35.000742 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:58:35.000742 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:58:35.002189 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:58:35.002189 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 26 14:58:35.002189 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:58:35.002189 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 26 14:58:35.002189 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 14:58:35.002189 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 14:58:35.002189 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 14:58:35.002189 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 14:58:35.002189 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 14:58:35.002189 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 14:58:35.002189 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 14:58:35.002189 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 26 14:58:35.002189 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 634/1249] 10.222.0.46 () {56 vars in 1096 bytes} [Thu May 26 14:58:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:58:35.006223 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4735f98b-7ca6-4795-8c00-4ad42b3ab678 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:58:35.123190 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4735f98b-7ca6-4795-8c00-4ad42b3ab678 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:58:35.160440 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:58:35.166298 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf2bdac-e92a-458d-9d55-668d439d0a13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:58:35.166987 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf2bdac-e92a-458d-9d55-668d439d0a13 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:58:35.167272 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf2bdac-e92a-458d-9d55-668d439d0a13 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:58:35.167561 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf2bdac-e92a-458d-9d55-668d439d0a13 None admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:58:35.168360 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf2bdac-e92a-458d-9d55-668d439d0a13 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:58:35.168632 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf2bdac-e92a-458d-9d55-668d439d0a13 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:58:35.170132 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf2bdac-e92a-458d-9d55-668d439d0a13 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:58:35.184292 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 616/1250] 10.222.0.46 () {56 vars in 1099 bytes} [Thu May 26 14:58:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:58:35.185587 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5418f4-482d-4981-9ce7-81d0a0034e5f None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:58:35.279428 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:58:35.311455 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5418f4-482d-4981-9ce7-81d0a0034e5f None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:58:35.362158 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffd148f-2528-4ab3-8549-aedd0c3be81d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:58:35.362975 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffd148f-2528-4ab3-8549-aedd0c3be81d None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:58:35.363239 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffd148f-2528-4ab3-8549-aedd0c3be81d None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:58:35.363520 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cffd148f-2528-4ab3-8549-aedd0c3be81d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:58:35.378772 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffd148f-2528-4ab3-8549-aedd0c3be81d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:58:35.380672 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cffd148f-2528-4ab3-8549-aedd0c3be81d None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:58:35.401811 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-cffd148f-2528-4ab3-8549-aedd0c3be81d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 14:58:35.401811 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:58:35.401811 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:58:35.401811 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:58:35.401811 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:58:35.401811 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:58:35.401811 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:58:35.401811 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:58:35.401811 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:58:35.401811 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:58:35.401811 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:58:35.401811 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:58:35.401811 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 14:58:35.401811 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 14:58:35.401811 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 14:58:35.402901 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 14:58:35.402901 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:58:35.402901 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:58:35.402901 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:58:35.402901 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:58:35.402901 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:58:35.402901 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:58:35.402901 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:58:35.402901 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 26 14:58:35.402901 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:58:35.402901 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 26 14:58:35.402901 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:58:35.402901 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:58:35.402901 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:58:35.402901 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 26 14:58:35.402901 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:58:35.402901 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 26 14:58:35.403804 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 14:58:35.403804 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 14:58:35.403804 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 14:58:35.403804 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 26 14:58:35.403804 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 635/1251] 10.222.0.46 () {56 vars in 1108 bytes} [Thu May 26 14:58:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:58:35.409438 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf2bdac-e92a-458d-9d55-668d439d0a13 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:58:35.522380 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf2bdac-e92a-458d-9d55-668d439d0a13 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:58:35.559418 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80e91b9-47fd-43f0-a6fe-cd5beaf57202 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:58:35.560314 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80e91b9-47fd-43f0-a6fe-cd5beaf57202 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:58:35.560594 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80e91b9-47fd-43f0-a6fe-cd5beaf57202 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:58:35.560865 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80e91b9-47fd-43f0-a6fe-cd5beaf57202 None admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:58:35.561604 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80e91b9-47fd-43f0-a6fe-cd5beaf57202 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:58:35.561907 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80e91b9-47fd-43f0-a6fe-cd5beaf57202 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:58:35.563314 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80e91b9-47fd-43f0-a6fe-cd5beaf57202 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:58:35.575356 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 617/1252] 10.222.0.46 () {56 vars in 1109 bytes} [Thu May 26 14:58:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 14:58:35.581045 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffd148f-2528-4ab3-8549-aedd0c3be81d None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:58:35.701511 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cffd148f-2528-4ab3-8549-aedd0c3be81d None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:58:35.729718 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:58:35.739876 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40149ff-0070-41b1-9ee8-a68682facf14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:58:35.740681 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40149ff-0070-41b1-9ee8-a68682facf14 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:58:35.740962 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40149ff-0070-41b1-9ee8-a68682facf14 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:58:35.741237 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40149ff-0070-41b1-9ee8-a68682facf14 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:58:35.749728 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40149ff-0070-41b1-9ee8-a68682facf14 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:58:35.751861 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40149ff-0070-41b1-9ee8-a68682facf14 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:58:35.765768 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-e40149ff-0070-41b1-9ee8-a68682facf14 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 26 14:58:35.765768 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:58:35.765768 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:58:35.765768 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:58:35.765768 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:58:35.765768 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:58:35.765768 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:58:35.765768 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:58:35.765768 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:58:35.765768 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:58:35.765768 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:58:35.765768 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:58:35.765768 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 14:58:35.765768 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 14:58:35.765768 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 14:58:35.765768 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 14:58:35.766901 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:58:35.766901 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:58:35.766901 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 14:58:35.766901 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 14:58:35.766901 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 14:58:35.766901 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 14:58:35.766901 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:58:35.766901 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:58:35.766901 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:58:35.766901 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:58:35.766901 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:58:35.766901 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 26 14:58:35.766901 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:58:35.766901 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 26 14:58:35.766901 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:58:35.766901 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:58:35.766901 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:58:35.766901 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 26 14:58:35.768258 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:58:35.768258 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 26 14:58:35.768258 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 14:58:35.768258 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 14:58:35.768258 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 14:58:35.768258 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 14:58:35.768258 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 14:58:35.768258 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 14:58:35.768258 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 26 14:58:35.768258 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 26 14:58:35.771879 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 636/1253] 10.222.0.46 () {56 vars in 1108 bytes} [Thu May 26 14:58:35 2022] GET /identity/v3/users/placement => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:58:35.772918 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80e91b9-47fd-43f0-a6fe-cd5beaf57202 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:58:35.833381 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:58:35.884813 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80e91b9-47fd-43f0-a6fe-cd5beaf57202 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:58:35.929484 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69b930a-af67-4587-b711-fd5377fdc2c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:58:35.930533 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69b930a-af67-4587-b711-fd5377fdc2c9 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:58:35.930866 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69b930a-af67-4587-b711-fd5377fdc2c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:58:35.931172 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69b930a-af67-4587-b711-fd5377fdc2c9 None admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:58:35.932179 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69b930a-af67-4587-b711-fd5377fdc2c9 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:58:35.932564 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69b930a-af67-4587-b711-fd5377fdc2c9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:58:35.934162 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69b930a-af67-4587-b711-fd5377fdc2c9 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:58:35.954539 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 618/1254] 10.222.0.46 () {56 vars in 1143 bytes} [Thu May 26 14:58:35 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 14:58:35.960963 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40149ff-0070-41b1-9ee8-a68682facf14 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:58:36.089375 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40149ff-0070-41b1-9ee8-a68682facf14 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:58:36.129381 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0f1568-1c7b-42f8-ab22-f39c63a4ba91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:58:36.130083 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0f1568-1c7b-42f8-ab22-f39c63a4ba91 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:58:36.130383 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0f1568-1c7b-42f8-ab22-f39c63a4ba91 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:58:36.130652 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0f1568-1c7b-42f8-ab22-f39c63a4ba91 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:58:36.191798 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:58:36.205015 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0f1568-1c7b-42f8-ab22-f39c63a4ba91 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:58:36.206549 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0f1568-1c7b-42f8-ab22-f39c63a4ba91 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:58:36.313889 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 637/1255] 10.222.0.46 () {58 vars in 1282 bytes} [Thu May 26 14:58:35 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 26 14:58:36.319251 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69b930a-af67-4587-b711-fd5377fdc2c9 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:58:36.430949 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69b930a-af67-4587-b711-fd5377fdc2c9 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:58:36.446817 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:58:36.471723 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8239f522-28a8-4997-846d-14c092a23daa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:58:36.472545 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8239f522-28a8-4997-846d-14c092a23daa None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:58:36.472830 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8239f522-28a8-4997-846d-14c092a23daa None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:58:36.473101 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8239f522-28a8-4997-846d-14c092a23daa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:58:36.484480 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8239f522-28a8-4997-846d-14c092a23daa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:58:36.486143 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8239f522-28a8-4997-846d-14c092a23daa None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:58:36.504557 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-8239f522-28a8-4997-846d-14c092a23daa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 14:58:36.504557 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:58:36.504557 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:58:36.504557 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:58:36.504557 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:58:36.504557 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:58:36.504557 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:58:36.504557 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:58:36.504557 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:58:36.504557 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:58:36.504557 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:58:36.504557 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:58:36.504557 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 14:58:36.504557 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 14:58:36.504557 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 14:58:36.505717 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 14:58:36.505717 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:58:36.505717 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:58:36.505717 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:58:36.505717 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:58:36.505717 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:58:36.505717 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:58:36.505717 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:58:36.505717 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 26 14:58:36.505717 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:58:36.505717 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 26 14:58:36.505717 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:58:36.505717 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:58:36.505717 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:58:36.505717 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 26 14:58:36.505717 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:58:36.505717 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 26 14:58:36.506903 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 14:58:36.506903 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 14:58:36.506903 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 14:58:36.506903 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 26 14:58:36.510424 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 619/1256] 10.222.0.46 () {56 vars in 1108 bytes} [Thu May 26 14:58:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:58:36.511425 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0f1568-1c7b-42f8-ab22-f39c63a4ba91 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:58:36.608144 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:58:36.621819 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0f1568-1c7b-42f8-ab22-f39c63a4ba91 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:58:36.661255 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aac0a40-39f0-489d-b910-8844cd3b140a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:58:36.662029 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aac0a40-39f0-489d-b910-8844cd3b140a None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:58:36.662370 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aac0a40-39f0-489d-b910-8844cd3b140a None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:58:36.662664 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aac0a40-39f0-489d-b910-8844cd3b140a None admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:58:36.663380 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aac0a40-39f0-489d-b910-8844cd3b140a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:58:36.663684 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aac0a40-39f0-489d-b910-8844cd3b140a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:58:36.665178 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aac0a40-39f0-489d-b910-8844cd3b140a None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:58:36.677262 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 638/1257] 10.222.0.46 () {56 vars in 1109 bytes} [Thu May 26 14:58:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:58:36.683450 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8239f522-28a8-4997-846d-14c092a23daa None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:58:36.797149 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8239f522-28a8-4997-846d-14c092a23daa None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:58:36.838131 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b905d0-3791-4f1a-9109-3b85aeebbb9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:58:36.839009 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b905d0-3791-4f1a-9109-3b85aeebbb9c None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:58:36.839334 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b905d0-3791-4f1a-9109-3b85aeebbb9c None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:58:36.839638 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b905d0-3791-4f1a-9109-3b85aeebbb9c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:58:36.847037 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b905d0-3791-4f1a-9109-3b85aeebbb9c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:58:36.848599 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b905d0-3791-4f1a-9109-3b85aeebbb9c None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:58:36.863471 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-65b905d0-3791-4f1a-9109-3b85aeebbb9c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 14:58:36.863471 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:58:36.863471 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:58:36.863471 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:58:36.863471 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:58:36.863471 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:58:36.863471 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:58:36.863471 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:58:36.863471 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:58:36.863471 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:58:36.863471 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:58:36.863471 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:58:36.863471 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 14:58:36.863471 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 14:58:36.863471 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 14:58:36.864671 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 14:58:36.864671 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:58:36.864671 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:58:36.864671 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:58:36.864671 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:58:36.864671 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:58:36.864671 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:58:36.864671 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:58:36.864671 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 26 14:58:36.864671 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:58:36.864671 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 26 14:58:36.864671 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:58:36.864671 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:58:36.864671 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:58:36.864671 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 26 14:58:36.864671 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:58:36.864671 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 26 14:58:36.865852 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 14:58:36.865852 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 14:58:36.865852 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 14:58:36.865852 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 14:58:36.865852 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 14:58:36.865852 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 14:58:36.865852 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 14:58:36.865852 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 26 14:58:36.865852 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 620/1258] 10.222.0.46 () {56 vars in 1111 bytes} [Thu May 26 14:58:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:58:36.871119 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aac0a40-39f0-489d-b910-8844cd3b140a None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:58:36.979427 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aac0a40-39f0-489d-b910-8844cd3b140a None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:58:37.015880 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2237c948-e41b-43f9-af9f-01800eea5b33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:58:37.016556 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2237c948-e41b-43f9-af9f-01800eea5b33 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:58:37.016846 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2237c948-e41b-43f9-af9f-01800eea5b33 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:58:37.017117 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2237c948-e41b-43f9-af9f-01800eea5b33 None admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:58:37.017940 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2237c948-e41b-43f9-af9f-01800eea5b33 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:58:37.018252 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2237c948-e41b-43f9-af9f-01800eea5b33 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:58:37.019689 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2237c948-e41b-43f9-af9f-01800eea5b33 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:58:37.037814 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 639/1259] 10.222.0.46 () {56 vars in 1148 bytes} [Thu May 26 14:58:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:58:37.039460 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b905d0-3791-4f1a-9109-3b85aeebbb9c None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:58:37.067330 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:58:37.146526 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b905d0-3791-4f1a-9109-3b85aeebbb9c None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:58:37.183527 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a406773-d973-489c-91d2-6ba220452923 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:58:37.184190 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a406773-d973-489c-91d2-6ba220452923 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:58:37.184488 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a406773-d973-489c-91d2-6ba220452923 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:58:37.184735 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a406773-d973-489c-91d2-6ba220452923 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:58:37.185602 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a406773-d973-489c-91d2-6ba220452923 None admin] RBAC: Adding query filter params (role.id=a2e551773cbd4df084020676865b8222, scope.project.id=a17fc8227d4c4da6b2847b350e912ee3, user.id=bf55a00e3b084aa3b22c25a90e6a773c) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:58:37.185914 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a406773-d973-489c-91d2-6ba220452923 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:58:37.187386 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a406773-d973-489c-91d2-6ba220452923 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:58:37.194274 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 621/1260] 10.222.0.46 () {56 vars in 1374 bytes} [Thu May 26 14:58:37 2022] GET /identity/v3/role_assignments?user.id=bf55a00e3b084aa3b22c25a90e6a773c&scope.project.id=a17fc8227d4c4da6b2847b350e912ee3&role.id=a2e551773cbd4df084020676865b8222 => generated 259 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:58:39.371252 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad54dd51-54c5-436f-b21c-84eef5f49870 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:58:39.371841 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad54dd51-54c5-436f-b21c-84eef5f49870 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:58:39.371989 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad54dd51-54c5-436f-b21c-84eef5f49870 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:58:39.373672 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 640/1261] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 14:58:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 14:58:39.380247 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca3ac06-2d30-41e8-9d97-256536f686c1 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:58:39.380775 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca3ac06-2d30-41e8-9d97-256536f686c1 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:58:39.380932 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca3ac06-2d30-41e8-9d97-256536f686c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:58:39.402301 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-8ca3ac06-2d30-41e8-9d97-256536f686c1 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:58:39.403661 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-8ca3ac06-2d30-41e8-9d97-256536f686c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:58:39.476159 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 622/1262] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 14:58:39 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 14:58:39.498273 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad54dd51-54c5-436f-b21c-84eef5f49870 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:58:39.509094 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:58:39.601381 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad54dd51-54c5-436f-b21c-84eef5f49870 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:58:39.640435 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f372a2-2511-4c36-9638-3a502a5f473c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:58:39.641316 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f372a2-2511-4c36-9638-3a502a5f473c None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:58:39.642616 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f372a2-2511-4c36-9638-3a502a5f473c None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:58:39.642902 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f372a2-2511-4c36-9638-3a502a5f473c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:58:39.653528 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f372a2-2511-4c36-9638-3a502a5f473c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 14:58:39.661189 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-12f372a2-2511-4c36-9638-3a502a5f473c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 14:58:39.661189 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:58:39.661189 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:58:39.661189 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:58:39.661189 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:58:39.661189 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:58:39.661189 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:58:39.661189 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:58:39.661189 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:58:39.661189 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:58:39.661189 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:58:39.661189 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:58:39.661189 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 14:58:39.661189 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 14:58:39.661189 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 14:58:39.661189 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 14:58:39.662290 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 14:58:39.662290 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 14:58:39.662290 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 14:58:39.662290 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 14:58:39.662290 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:58:39.662290 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:58:39.662290 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:58:39.662290 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:58:39.662290 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:58:39.662290 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:58:39.662290 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:58:39.662290 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 26 14:58:39.662290 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:58:39.662290 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 26 14:58:39.662290 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:58:39.662290 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:58:39.663328 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:58:39.663328 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 26 14:58:39.663328 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:58:39.663328 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 26 14:58:39.663328 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 14:58:39.663328 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 14:58:39.663328 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 14:58:39.663328 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 14:58:39.663328 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 14:58:39.663328 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 14:58:39.663328 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 14:58:39.663328 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 26 14:58:39.663328 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 641/1263] 10.222.0.46 () {56 vars in 1096 bytes} [Thu May 26 14:58:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:58:39.668379 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca3ac06-2d30-41e8-9d97-256536f686c1 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:58:39.686497 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:58:39.767940 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca3ac06-2d30-41e8-9d97-256536f686c1 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:58:39.800211 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722451d8-7f5d-40ea-ba06-33b5e80d3e82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:58:39.800866 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722451d8-7f5d-40ea-ba06-33b5e80d3e82 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:58:39.801148 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722451d8-7f5d-40ea-ba06-33b5e80d3e82 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:58:39.801425 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722451d8-7f5d-40ea-ba06-33b5e80d3e82 None admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:58:39.802191 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722451d8-7f5d-40ea-ba06-33b5e80d3e82 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:58:39.802496 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722451d8-7f5d-40ea-ba06-33b5e80d3e82 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:58:39.803898 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722451d8-7f5d-40ea-ba06-33b5e80d3e82 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:58:39.811666 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 623/1264] 10.222.0.46 () {56 vars in 1099 bytes} [Thu May 26 14:58:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:58:39.817981 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f372a2-2511-4c36-9638-3a502a5f473c None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:58:39.882382 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:58:39.915423 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f372a2-2511-4c36-9638-3a502a5f473c None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:58:39.947772 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5e453f-e1a1-4f85-859c-b69c4ee0e465 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:58:39.948552 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5e453f-e1a1-4f85-859c-b69c4ee0e465 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:58:39.948835 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5e453f-e1a1-4f85-859c-b69c4ee0e465 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:58:39.949110 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5e453f-e1a1-4f85-859c-b69c4ee0e465 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:58:39.959888 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5e453f-e1a1-4f85-859c-b69c4ee0e465 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:58:39.961319 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5e453f-e1a1-4f85-859c-b69c4ee0e465 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:58:39.977819 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-3a5e453f-e1a1-4f85-859c-b69c4ee0e465 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 14:58:39.977819 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:58:39.977819 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:58:39.977819 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:58:39.977819 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:58:39.977819 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:58:39.977819 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:58:39.977819 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:58:39.977819 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:58:39.977819 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:58:39.977819 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:58:39.977819 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:58:39.977819 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 14:58:39.977819 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 14:58:39.977819 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 14:58:39.978878 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 14:58:39.978878 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:58:39.978878 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:58:39.978878 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:58:39.978878 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:58:39.978878 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:58:39.978878 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:58:39.978878 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:58:39.978878 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 26 14:58:39.978878 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:58:39.978878 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 26 14:58:39.978878 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:58:39.978878 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:58:39.978878 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:58:39.978878 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 26 14:58:39.978878 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:58:39.978878 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 26 14:58:39.980105 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 14:58:39.980105 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 14:58:39.980105 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 14:58:39.980105 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 26 14:58:39.980105 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 642/1265] 10.222.0.46 () {56 vars in 1108 bytes} [Thu May 26 14:58:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:58:39.984575 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722451d8-7f5d-40ea-ba06-33b5e80d3e82 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:58:40.062588 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:58:40.085211 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722451d8-7f5d-40ea-ba06-33b5e80d3e82 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:58:40.117844 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d211d9b3-af1f-4a44-a554-5bb243d6d97f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:58:40.118497 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d211d9b3-af1f-4a44-a554-5bb243d6d97f None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:58:40.118781 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d211d9b3-af1f-4a44-a554-5bb243d6d97f None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:58:40.119059 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d211d9b3-af1f-4a44-a554-5bb243d6d97f None admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:58:40.119774 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d211d9b3-af1f-4a44-a554-5bb243d6d97f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:58:40.120082 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d211d9b3-af1f-4a44-a554-5bb243d6d97f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:58:40.121443 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d211d9b3-af1f-4a44-a554-5bb243d6d97f None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:58:40.133619 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 624/1266] 10.222.0.46 () {56 vars in 1109 bytes} [Thu May 26 14:58:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:58:40.139146 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5e453f-e1a1-4f85-859c-b69c4ee0e465 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:58:40.236338 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5e453f-e1a1-4f85-859c-b69c4ee0e465 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:58:40.255620 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:58:40.269260 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a38832-489d-440d-9d21-985e207f897a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:58:40.270075 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a38832-489d-440d-9d21-985e207f897a None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:58:40.270359 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a38832-489d-440d-9d21-985e207f897a None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:58:40.270632 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a38832-489d-440d-9d21-985e207f897a None admin] PATH_INFO: `/v3/users/placement` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:58:40.277713 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a38832-489d-440d-9d21-985e207f897a None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:58:40.279132 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a38832-489d-440d-9d21-985e207f897a None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:58:40.292807 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-86a38832-489d-440d-9d21-985e207f897a None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 26 14:58:40.292807 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:58:40.292807 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:58:40.292807 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:58:40.292807 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:58:40.292807 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:58:40.292807 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:58:40.292807 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:58:40.292807 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:58:40.292807 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:58:40.292807 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:58:40.292807 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:58:40.292807 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 14:58:40.292807 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 14:58:40.292807 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 14:58:40.292807 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 14:58:40.293935 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:58:40.293935 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:58:40.293935 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 14:58:40.293935 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 14:58:40.293935 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 14:58:40.293935 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 14:58:40.293935 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:58:40.293935 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:58:40.293935 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:58:40.293935 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:58:40.293935 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:58:40.293935 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 26 14:58:40.293935 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:58:40.293935 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 26 14:58:40.293935 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:58:40.293935 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:58:40.293935 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:58:40.293935 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 26 14:58:40.295014 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:58:40.295014 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 26 14:58:40.295014 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 14:58:40.295014 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 14:58:40.295014 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 14:58:40.295014 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 14:58:40.295014 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 14:58:40.295014 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 14:58:40.295014 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 26 14:58:40.295014 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 26 14:58:40.295014 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 643/1267] 10.222.0.46 () {56 vars in 1108 bytes} [Thu May 26 14:58:40 2022] GET /identity/v3/users/placement => generated 87 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:58:40.300183 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d211d9b3-af1f-4a44-a554-5bb243d6d97f None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:58:40.402369 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d211d9b3-af1f-4a44-a554-5bb243d6d97f None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:58:40.440603 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb35f95-a3e7-470f-80b1-3a928afb70fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:58:40.441276 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb35f95-a3e7-470f-80b1-3a928afb70fd None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:58:40.441594 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb35f95-a3e7-470f-80b1-3a928afb70fd None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:58:40.441877 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb35f95-a3e7-470f-80b1-3a928afb70fd None admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:58:40.442710 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb35f95-a3e7-470f-80b1-3a928afb70fd None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:58:40.443022 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb35f95-a3e7-470f-80b1-3a928afb70fd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:58:40.444459 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb35f95-a3e7-470f-80b1-3a928afb70fd None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:58:40.452951 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:58:40.462723 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 625/1268] 10.222.0.46 () {56 vars in 1143 bytes} [Thu May 26 14:58:40 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:58:40.468275 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a38832-489d-440d-9d21-985e207f897a None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:58:40.568416 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a38832-489d-440d-9d21-985e207f897a None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:58:40.601771 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38223aa5-8660-401a-be7f-d06581a6dac8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:58:40.602661 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38223aa5-8660-401a-be7f-d06581a6dac8 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:58:40.602945 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38223aa5-8660-401a-be7f-d06581a6dac8 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:58:40.603220 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38223aa5-8660-401a-be7f-d06581a6dac8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:58:40.614830 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38223aa5-8660-401a-be7f-d06581a6dac8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:58:40.616269 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38223aa5-8660-401a-be7f-d06581a6dac8 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:58:40.633954 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-38223aa5-8660-401a-be7f-d06581a6dac8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 14:58:40.633954 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:58:40.633954 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:58:40.633954 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:58:40.633954 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:58:40.633954 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:58:40.633954 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:58:40.633954 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:58:40.633954 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:58:40.633954 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:58:40.633954 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:58:40.633954 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:58:40.633954 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 14:58:40.633954 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 14:58:40.633954 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 14:58:40.635023 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 14:58:40.635023 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:58:40.635023 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:58:40.635023 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:58:40.635023 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:58:40.635023 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:58:40.635023 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:58:40.635023 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:58:40.635023 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 26 14:58:40.635023 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:58:40.635023 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 26 14:58:40.635023 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:58:40.635023 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:58:40.635023 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:58:40.635023 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 26 14:58:40.635023 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:58:40.635023 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 26 14:58:40.636108 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 14:58:40.636108 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 14:58:40.636108 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 14:58:40.636108 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 26 14:58:40.636108 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 644/1269] 10.222.0.46 () {56 vars in 1108 bytes} [Thu May 26 14:58:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:58:40.641579 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb35f95-a3e7-470f-80b1-3a928afb70fd None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:58:40.740310 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb35f95-a3e7-470f-80b1-3a928afb70fd None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:58:40.773820 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd05be9-a3dc-4694-8118-2ca6176c985d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:58:40.774504 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd05be9-a3dc-4694-8118-2ca6176c985d None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:58:40.774853 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd05be9-a3dc-4694-8118-2ca6176c985d None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:58:40.775129 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd05be9-a3dc-4694-8118-2ca6176c985d None admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:58:40.775844 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd05be9-a3dc-4694-8118-2ca6176c985d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:58:40.776173 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd05be9-a3dc-4694-8118-2ca6176c985d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:58:40.777550 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd05be9-a3dc-4694-8118-2ca6176c985d None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:58:40.789581 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 626/1270] 10.222.0.46 () {56 vars in 1109 bytes} [Thu May 26 14:58:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:58:40.795214 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38223aa5-8660-401a-be7f-d06581a6dac8 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:58:40.825838 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:58:40.893722 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38223aa5-8660-401a-be7f-d06581a6dac8 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:58:40.928527 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28dde15f-14a2-43d4-a309-11c0761c41a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:58:40.929317 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dde15f-14a2-43d4-a309-11c0761c41a5 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:58:40.929631 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dde15f-14a2-43d4-a309-11c0761c41a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:58:40.929913 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28dde15f-14a2-43d4-a309-11c0761c41a5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:58:40.936439 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28dde15f-14a2-43d4-a309-11c0761c41a5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:58:40.937913 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28dde15f-14a2-43d4-a309-11c0761c41a5 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:58:40.952641 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-28dde15f-14a2-43d4-a309-11c0761c41a5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 14:58:40.952641 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:58:40.952641 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:58:40.952641 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:58:40.952641 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:58:40.952641 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:58:40.952641 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:58:40.952641 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:58:40.952641 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:58:40.952641 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:58:40.952641 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:58:40.952641 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:58:40.952641 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 14:58:40.952641 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 14:58:40.952641 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 14:58:40.954211 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 14:58:40.954211 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:58:40.954211 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:58:40.954211 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:58:40.954211 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:58:40.954211 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:58:40.954211 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:58:40.954211 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:58:40.954211 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 26 14:58:40.954211 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:58:40.954211 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 26 14:58:40.954211 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:58:40.954211 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:58:40.954211 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:58:40.954211 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 26 14:58:40.954211 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:58:40.954211 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 26 14:58:40.955384 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 14:58:40.955384 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 14:58:40.955384 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 14:58:40.955384 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 14:58:40.955384 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 14:58:40.955384 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 14:58:40.955384 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 14:58:40.955384 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 26 14:58:40.955384 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 645/1271] 10.222.0.46 () {56 vars in 1111 bytes} [Thu May 26 14:58:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:58:40.960319 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd05be9-a3dc-4694-8118-2ca6176c985d None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:58:41.018044 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:58:41.058414 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd05be9-a3dc-4694-8118-2ca6176c985d None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:58:41.091041 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbaa74d-68e9-4984-9d59-ea7ca7e4ffc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:58:41.091692 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbaa74d-68e9-4984-9d59-ea7ca7e4ffc2 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:58:41.091975 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbaa74d-68e9-4984-9d59-ea7ca7e4ffc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:58:41.092254 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbaa74d-68e9-4984-9d59-ea7ca7e4ffc2 None admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:58:41.093030 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbaa74d-68e9-4984-9d59-ea7ca7e4ffc2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:58:41.093337 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbaa74d-68e9-4984-9d59-ea7ca7e4ffc2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:58:41.094852 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcbaa74d-68e9-4984-9d59-ea7ca7e4ffc2 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:58:41.106522 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 627/1272] 10.222.0.46 () {56 vars in 1148 bytes} [Thu May 26 14:58:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:58:41.112412 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28dde15f-14a2-43d4-a309-11c0761c41a5 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:58:41.200871 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:58:41.210802 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28dde15f-14a2-43d4-a309-11c0761c41a5 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:58:41.243675 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec92d29c-add5-4b80-aa5c-63994b2ae8ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:58:41.244553 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec92d29c-add5-4b80-aa5c-63994b2ae8ad None admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:58:41.244833 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec92d29c-add5-4b80-aa5c-63994b2ae8ad None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:58:41.245103 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec92d29c-add5-4b80-aa5c-63994b2ae8ad None admin] PATH_INFO: `/v3/projects/a17fc8227d4c4da6b2847b350e912ee3/users/bf55a00e3b084aa3b22c25a90e6a773c/roles/a2e551773cbd4df084020676865b8222` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:58:41.247064 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec92d29c-add5-4b80-aa5c-63994b2ae8ad None admin] RBAC: Authorizing `identity:create_grant(project_id=a17fc8227d4c4da6b2847b350e912ee3, user_id=bf55a00e3b084aa3b22c25a90e6a773c, role_id=a2e551773cbd4df084020676865b8222)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:58:41.248487 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec92d29c-add5-4b80-aa5c-63994b2ae8ad None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:58:41.261563 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 646/1273] 10.222.0.46 () {58 vars in 1439 bytes} [Thu May 26 14:58:41 2022] PUT /identity/v3/projects/a17fc8227d4c4da6b2847b350e912ee3/users/bf55a00e3b084aa3b22c25a90e6a773c/roles/a2e551773cbd4df084020676865b8222 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 14:58:43.450915 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c585ff00-0997-4d94-b314-7c3e02f349eb None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:58:43.451476 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c585ff00-0997-4d94-b314-7c3e02f349eb None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:58:43.451476 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c585ff00-0997-4d94-b314-7c3e02f349eb None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:58:43.452836 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 628/1274] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 14:58:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 14:58:43.458018 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e8b3dc-fa38-4cf4-b584-005ce32491d9 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:58:43.458385 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e8b3dc-fa38-4cf4-b584-005ce32491d9 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:58:43.458585 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e8b3dc-fa38-4cf4-b584-005ce32491d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:58:43.475423 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-73e8b3dc-fa38-4cf4-b584-005ce32491d9 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:58:43.476601 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-73e8b3dc-fa38-4cf4-b584-005ce32491d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:58:43.541607 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 647/1275] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 14:58:43 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 14:58:43.565761 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c585ff00-0997-4d94-b314-7c3e02f349eb None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:58:43.664048 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c585ff00-0997-4d94-b314-7c3e02f349eb None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:58:43.681935 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:58:43.698161 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972a7eac-f06a-4be5-8d84-659f21854126 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:58:43.698954 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972a7eac-f06a-4be5-8d84-659f21854126 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:58:43.699242 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972a7eac-f06a-4be5-8d84-659f21854126 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:58:43.699519 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972a7eac-f06a-4be5-8d84-659f21854126 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:58:43.709909 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972a7eac-f06a-4be5-8d84-659f21854126 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 26 14:58:43.717598 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-972a7eac-f06a-4be5-8d84-659f21854126 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 26 14:58:43.717598 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:58:43.717598 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:58:43.717598 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:58:43.717598 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:58:43.717598 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:58:43.717598 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:58:43.717598 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:58:43.717598 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:58:43.717598 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:58:43.717598 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:58:43.717598 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:58:43.717598 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 26 14:58:43.717598 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(role_id) May 26 14:58:43.717598 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 26 14:58:43.717598 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 26 14:58:43.718677 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 26 14:58:43.718677 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 26 14:58:43.718677 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 26 14:58:43.718677 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 26 14:58:43.718677 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:58:43.718677 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:58:43.718677 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:58:43.718677 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:58:43.718677 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:58:43.718677 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:58:43.718677 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:58:43.718677 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 26 14:58:43.718677 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:58:43.718677 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 26 14:58:43.718677 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:58:43.718677 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:58:43.720385 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:58:43.720385 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 26 14:58:43.720385 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:58:43.720385 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 26 14:58:43.720385 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 26 14:58:43.720385 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 26 14:58:43.720385 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 26 14:58:43.720385 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 26 14:58:43.720385 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 26 14:58:43.720385 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 26 14:58:43.720385 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 26 14:58:43.720385 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 26 14:58:43.720385 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 629/1276] 10.222.0.46 () {56 vars in 1096 bytes} [Thu May 26 14:58:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:58:43.725286 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e8b3dc-fa38-4cf4-b584-005ce32491d9 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:58:43.803098 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:58:43.823232 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e8b3dc-fa38-4cf4-b584-005ce32491d9 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:58:43.858025 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4d732f-9226-4188-a5b2-a4b13adacdc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:58:43.858690 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4d732f-9226-4188-a5b2-a4b13adacdc7 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:58:43.858976 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4d732f-9226-4188-a5b2-a4b13adacdc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:58:43.859253 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4d732f-9226-4188-a5b2-a4b13adacdc7 None admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:58:43.859993 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4d732f-9226-4188-a5b2-a4b13adacdc7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:58:43.860300 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4d732f-9226-4188-a5b2-a4b13adacdc7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:58:43.861699 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4d732f-9226-4188-a5b2-a4b13adacdc7 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:58:43.869526 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 648/1277] 10.222.0.46 () {56 vars in 1099 bytes} [Thu May 26 14:58:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:58:43.875175 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972a7eac-f06a-4be5-8d84-659f21854126 None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:58:43.972999 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972a7eac-f06a-4be5-8d84-659f21854126 None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:58:44.004979 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160575f2-fa19-4101-8066-0c348d4e820f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:58:44.005912 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160575f2-fa19-4101-8066-0c348d4e820f None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:58:44.006198 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160575f2-fa19-4101-8066-0c348d4e820f None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:58:44.006475 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160575f2-fa19-4101-8066-0c348d4e820f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:58:44.017527 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160575f2-fa19-4101-8066-0c348d4e820f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:58:44.018970 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160575f2-fa19-4101-8066-0c348d4e820f None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:58:44.036214 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-160575f2-fa19-4101-8066-0c348d4e820f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 14:58:44.036214 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:58:44.036214 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:58:44.036214 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:58:44.036214 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:58:44.036214 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:58:44.036214 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:58:44.036214 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:58:44.036214 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:58:44.036214 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:58:44.036214 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:58:44.036214 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:58:44.036214 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 14:58:44.036214 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 14:58:44.036214 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 14:58:44.037246 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 14:58:44.037246 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:58:44.037246 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:58:44.037246 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:58:44.037246 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:58:44.037246 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:58:44.037246 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:58:44.037246 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:58:44.037246 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 26 14:58:44.037246 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:58:44.037246 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 26 14:58:44.037246 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:58:44.037246 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:58:44.037246 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:58:44.037246 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 26 14:58:44.037246 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:58:44.037246 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 26 14:58:44.038338 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 14:58:44.038338 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 14:58:44.038338 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 14:58:44.038338 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 26 14:58:44.038338 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 630/1278] 10.222.0.46 () {56 vars in 1108 bytes} [Thu May 26 14:58:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:58:44.043098 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4d732f-9226-4188-a5b2-a4b13adacdc7 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:58:44.140397 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4d732f-9226-4188-a5b2-a4b13adacdc7 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:58:44.173807 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9cf8d7-3ec3-46d3-a951-d67b7c607dba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:58:44.174467 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9cf8d7-3ec3-46d3-a951-d67b7c607dba None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:58:44.174748 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9cf8d7-3ec3-46d3-a951-d67b7c607dba None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:58:44.175021 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9cf8d7-3ec3-46d3-a951-d67b7c607dba None admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:58:44.175729 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9cf8d7-3ec3-46d3-a951-d67b7c607dba None admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:58:44.176032 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9cf8d7-3ec3-46d3-a951-d67b7c607dba None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:58:44.177441 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c9cf8d7-3ec3-46d3-a951-d67b7c607dba None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:58:44.188894 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 649/1279] 10.222.0.46 () {56 vars in 1109 bytes} [Thu May 26 14:58:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:58:44.194565 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160575f2-fa19-4101-8066-0c348d4e820f None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:58:44.225000 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:58:44.292890 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160575f2-fa19-4101-8066-0c348d4e820f None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:58:44.325243 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c3ccf3-ee55-4fff-a43f-98c1cdac623e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:58:44.326062 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c3ccf3-ee55-4fff-a43f-98c1cdac623e None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:58:44.326347 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c3ccf3-ee55-4fff-a43f-98c1cdac623e None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:58:44.326627 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c3ccf3-ee55-4fff-a43f-98c1cdac623e None admin] PATH_INFO: `/v3/users/placement` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:58:44.333842 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c3ccf3-ee55-4fff-a43f-98c1cdac623e None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:58:44.335286 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c3ccf3-ee55-4fff-a43f-98c1cdac623e None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:58:44.349804 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-80c3ccf3-ee55-4fff-a43f-98c1cdac623e None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 26 14:58:44.349804 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:58:44.349804 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:58:44.349804 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:58:44.349804 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:58:44.349804 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:58:44.349804 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:58:44.349804 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:58:44.349804 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:58:44.349804 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:58:44.349804 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:58:44.349804 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:58:44.349804 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 26 14:58:44.349804 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_user(user_id) May 26 14:58:44.349804 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 26 14:58:44.349804 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 26 14:58:44.350906 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:58:44.350906 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:58:44.350906 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 26 14:58:44.350906 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 14:58:44.350906 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 26 14:58:44.350906 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 26 14:58:44.350906 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:58:44.350906 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:58:44.350906 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:58:44.350906 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:58:44.350906 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:58:44.350906 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 26 14:58:44.350906 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:58:44.350906 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 26 14:58:44.350906 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:58:44.350906 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:58:44.350906 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:58:44.350906 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 26 14:58:44.352175 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:58:44.352175 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 26 14:58:44.352175 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 26 14:58:44.352175 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 26 14:58:44.352175 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 26 14:58:44.352175 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 26 14:58:44.352175 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 26 14:58:44.352175 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 26 14:58:44.352175 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 26 14:58:44.352175 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 26 14:58:44.352175 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 631/1280] 10.222.0.46 () {56 vars in 1108 bytes} [Thu May 26 14:58:44 2022] GET /identity/v3/users/placement => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:58:44.356759 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9cf8d7-3ec3-46d3-a951-d67b7c607dba None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:58:44.361040 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:58:44.466560 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9cf8d7-3ec3-46d3-a951-d67b7c607dba None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:58:44.499645 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c625b578-a84b-4d7e-8a59-4ad991f3551f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:58:44.500246 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c625b578-a84b-4d7e-8a59-4ad991f3551f None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:58:44.500557 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c625b578-a84b-4d7e-8a59-4ad991f3551f None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:58:44.500839 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c625b578-a84b-4d7e-8a59-4ad991f3551f None admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:58:44.501961 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c625b578-a84b-4d7e-8a59-4ad991f3551f None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:58:44.502275 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c625b578-a84b-4d7e-8a59-4ad991f3551f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:58:44.503698 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c625b578-a84b-4d7e-8a59-4ad991f3551f None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:58:44.521532 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 650/1281] 10.222.0.46 () {56 vars in 1143 bytes} [Thu May 26 14:58:44 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:58:44.527248 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c3ccf3-ee55-4fff-a43f-98c1cdac623e None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:58:44.615696 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:58:44.625566 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c3ccf3-ee55-4fff-a43f-98c1cdac623e None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:58:44.659835 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b629d792-8fb9-4f30-943b-f347ee24a1fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:58:44.660482 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b629d792-8fb9-4f30-943b-f347ee24a1fd None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:58:44.660769 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b629d792-8fb9-4f30-943b-f347ee24a1fd None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:58:44.661050 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b629d792-8fb9-4f30-943b-f347ee24a1fd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:58:44.735550 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b629d792-8fb9-4f30-943b-f347ee24a1fd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:58:44.737022 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b629d792-8fb9-4f30-943b-f347ee24a1fd None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:58:44.824547 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:58:44.836212 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 632/1282] 10.222.0.46 () {58 vars in 1282 bytes} [Thu May 26 14:58:44 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 26 14:58:44.842052 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c625b578-a84b-4d7e-8a59-4ad991f3551f None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:58:44.902387 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:58:44.940125 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c625b578-a84b-4d7e-8a59-4ad991f3551f None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:58:44.973230 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60159eeb-0fc1-4464-87af-1ca940afd0ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:58:44.974038 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60159eeb-0fc1-4464-87af-1ca940afd0ce None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:58:44.974323 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60159eeb-0fc1-4464-87af-1ca940afd0ce None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:58:44.974600 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60159eeb-0fc1-4464-87af-1ca940afd0ce None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:58:44.984308 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60159eeb-0fc1-4464-87af-1ca940afd0ce None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:58:44.985769 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60159eeb-0fc1-4464-87af-1ca940afd0ce None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:58:45.003786 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-60159eeb-0fc1-4464-87af-1ca940afd0ce None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 26 14:58:45.003786 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:58:45.003786 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:58:45.003786 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:58:45.003786 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:58:45.003786 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:58:45.003786 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:58:45.003786 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:58:45.003786 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:58:45.003786 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:58:45.003786 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:58:45.003786 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:58:45.003786 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 26 14:58:45.003786 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 26 14:58:45.003786 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 26 14:58:45.004841 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 26 14:58:45.004841 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:58:45.004841 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:58:45.004841 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:58:45.004841 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:58:45.004841 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:58:45.004841 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:58:45.004841 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:58:45.004841 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 26 14:58:45.004841 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:58:45.004841 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 26 14:58:45.004841 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:58:45.004841 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:58:45.004841 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:58:45.004841 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 26 14:58:45.004841 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:58:45.004841 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 26 14:58:45.006031 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 26 14:58:45.006031 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 26 14:58:45.006031 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 26 14:58:45.006031 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 26 14:58:45.006031 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 651/1283] 10.222.0.46 () {56 vars in 1108 bytes} [Thu May 26 14:58:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:58:45.010761 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b629d792-8fb9-4f30-943b-f347ee24a1fd None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:58:45.108659 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b629d792-8fb9-4f30-943b-f347ee24a1fd None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:58:45.142648 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f0a7dd-ddaf-40c5-bade-357cd0a2b37d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:58:45.143301 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f0a7dd-ddaf-40c5-bade-357cd0a2b37d None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:58:45.143580 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f0a7dd-ddaf-40c5-bade-357cd0a2b37d None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:58:45.143854 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f0a7dd-ddaf-40c5-bade-357cd0a2b37d None admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:58:45.144559 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f0a7dd-ddaf-40c5-bade-357cd0a2b37d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:58:45.144861 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f0a7dd-ddaf-40c5-bade-357cd0a2b37d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:58:45.146240 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f0a7dd-ddaf-40c5-bade-357cd0a2b37d None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:58:45.157277 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 633/1284] 10.222.0.46 () {56 vars in 1109 bytes} [Thu May 26 14:58:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 14:58:45.163046 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60159eeb-0fc1-4464-87af-1ca940afd0ce None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:58:45.260591 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60159eeb-0fc1-4464-87af-1ca940afd0ce None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:58:45.275864 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:58:45.293763 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd07764a-0fb6-44bf-9b01-522902e77eb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:58:45.294546 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd07764a-0fb6-44bf-9b01-522902e77eb1 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:58:45.294830 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd07764a-0fb6-44bf-9b01-522902e77eb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:58:45.295108 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd07764a-0fb6-44bf-9b01-522902e77eb1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:58:45.302097 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd07764a-0fb6-44bf-9b01-522902e77eb1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:58:45.303527 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd07764a-0fb6-44bf-9b01-522902e77eb1 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:58:45.317838 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application [None req-bd07764a-0fb6-44bf-9b01-522902e77eb1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 26 14:58:45.317838 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:58:45.317838 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:58:45.317838 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:58:45.317838 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:58:45.317838 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:58:45.317838 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:58:45.317838 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:58:45.317838 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:58:45.317838 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:58:45.317838 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:58:45.317838 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:58:45.317838 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 26 14:58:45.317838 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_project(project_id) May 26 14:58:45.317838 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 26 14:58:45.319841 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 26 14:58:45.319841 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:58:45.319841 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:58:45.319841 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:58:45.319841 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:58:45.319841 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:58:45.319841 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:58:45.319841 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:58:45.319841 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application with Lock( May 26 14:58:45.319841 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:58:45.319841 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._enter() May 26 14:58:45.319841 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:58:45.319841 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:58:45.319841 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:58:45.319841 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.creator() May 26 14:58:45.319841 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:58:45.319841 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application created_value = creator( May 26 14:58:45.320731 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 26 14:58:45.320731 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 26 14:58:45.320731 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 26 14:58:45.320731 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 26 14:58:45.320731 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 26 14:58:45.320731 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 26 14:58:45.320731 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 26 14:58:45.320731 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: ERROR keystone.server.flask.application May 26 14:58:45.320731 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 652/1285] 10.222.0.46 () {56 vars in 1111 bytes} [Thu May 26 14:58:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:58:45.324987 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f0a7dd-ddaf-40c5-bade-357cd0a2b37d None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:58:45.386476 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:58:45.440112 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f0a7dd-ddaf-40c5-bade-357cd0a2b37d None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:58:45.472969 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ac2e57-6d76-470f-bc8b-da446e0b34fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:58:45.473653 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ac2e57-6d76-470f-bc8b-da446e0b34fc None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:58:45.473940 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ac2e57-6d76-470f-bc8b-da446e0b34fc None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:58:45.474218 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ac2e57-6d76-470f-bc8b-da446e0b34fc None admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:58:45.474997 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ac2e57-6d76-470f-bc8b-da446e0b34fc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:58:45.475310 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ac2e57-6d76-470f-bc8b-da446e0b34fc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:58:45.476754 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ac2e57-6d76-470f-bc8b-da446e0b34fc None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:58:45.488413 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 634/1286] 10.222.0.46 () {56 vars in 1148 bytes} [Thu May 26 14:58:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:58:45.494362 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd07764a-0fb6-44bf-9b01-522902e77eb1 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:58:45.592552 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd07764a-0fb6-44bf-9b01-522902e77eb1 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:58:45.624812 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b69ca7-c4d7-448b-9ad0-00c3655dc15b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:58:45.626257 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b69ca7-c4d7-448b-9ad0-00c3655dc15b None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:58:45.626538 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b69ca7-c4d7-448b-9ad0-00c3655dc15b None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:58:45.626810 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b69ca7-c4d7-448b-9ad0-00c3655dc15b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:58:45.627626 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b69ca7-c4d7-448b-9ad0-00c3655dc15b None admin] RBAC: Adding query filter params (role.id=a2e551773cbd4df084020676865b8222, scope.project.id=a17fc8227d4c4da6b2847b350e912ee3, user.id=bf55a00e3b084aa3b22c25a90e6a773c) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:58:45.627935 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b69ca7-c4d7-448b-9ad0-00c3655dc15b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:58:45.629349 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b69ca7-c4d7-448b-9ad0-00c3655dc15b None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:58:45.635079 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 653/1287] 10.222.0.46 () {56 vars in 1374 bytes} [Thu May 26 14:58:45 2022] GET /identity/v3/role_assignments?user.id=bf55a00e3b084aa3b22c25a90e6a773c&scope.project.id=a17fc8227d4c4da6b2847b350e912ee3&role.id=a2e551773cbd4df084020676865b8222 => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:58:47.829200 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9579b53-48a4-4d46-8f77-36f0ebcff8e9 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:58:47.829784 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9579b53-48a4-4d46-8f77-36f0ebcff8e9 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:58:47.829784 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9579b53-48a4-4d46-8f77-36f0ebcff8e9 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:58:47.831154 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 635/1288] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 14:58:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 14:58:47.836486 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abeb547-d453-4bc2-b3c5-3c5dc15114c3 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:58:47.836890 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abeb547-d453-4bc2-b3c5-3c5dc15114c3 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:58:47.837054 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abeb547-d453-4bc2-b3c5-3c5dc15114c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:58:47.854571 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-5abeb547-d453-4bc2-b3c5-3c5dc15114c3 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:58:47.855759 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-5abeb547-d453-4bc2-b3c5-3c5dc15114c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:58:47.884962 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:58:47.921697 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 654/1289] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 14:58:47 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 14:58:47.944050 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9579b53-48a4-4d46-8f77-36f0ebcff8e9 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:58:48.043390 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9579b53-48a4-4d46-8f77-36f0ebcff8e9 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:58:48.053771 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:58:48.077140 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e8ab8f-7a71-449f-bfac-1b01b35d7d1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:58:48.077958 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e8ab8f-7a71-449f-bfac-1b01b35d7d1e None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:58:48.078808 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e8ab8f-7a71-449f-bfac-1b01b35d7d1e None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:58:48.079101 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e8ab8f-7a71-449f-bfac-1b01b35d7d1e None admin] PATH_INFO: `/v3/services/placement` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:58:48.079747 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e8ab8f-7a71-449f-bfac-1b01b35d7d1e None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:58:48.081124 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e8ab8f-7a71-449f-bfac-1b01b35d7d1e None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:58:48.092749 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-f4e8ab8f-7a71-449f-bfac-1b01b35d7d1e None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 26 14:58:48.092749 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:58:48.092749 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 14:58:48.092749 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 14:58:48.092749 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 14:58:48.092749 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 14:58:48.092749 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 14:58:48.092749 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 14:58:48.092749 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 26 14:58:48.092749 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 26 14:58:48.092749 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 14:58:48.092749 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 26 14:58:48.092749 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:58:48.092749 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:58:48.092749 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:58:48.092749 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:58:48.092749 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:58:48.094027 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:58:48.094027 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:58:48.094027 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:58:48.094027 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:58:48.094027 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:58:48.094027 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:58:48.094027 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 14:58:48.094027 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 14:58:48.094027 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 14:58:48.094027 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 14:58:48.094027 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:58:48.094027 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:58:48.094027 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:58:48.094027 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:58:48.094027 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:58:48.094027 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:58:48.095115 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:58:48.095115 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 26 14:58:48.095115 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:58:48.095115 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 26 14:58:48.095115 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:58:48.095115 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:58:48.095115 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:58:48.095115 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 26 14:58:48.095115 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:58:48.095115 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 26 14:58:48.095115 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 14:58:48.095115 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 14:58:48.095115 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 26 14:58:48.095115 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 26 14:58:48.095115 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 636/1290] 10.222.0.46 () {56 vars in 1117 bytes} [Thu May 26 14:58:47 2022] GET /identity/v3/services/placement => generated 90 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:58:48.100089 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abeb547-d453-4bc2-b3c5-3c5dc15114c3 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:58:48.196345 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abeb547-d453-4bc2-b3c5-3c5dc15114c3 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:58:48.229087 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1a69b0-bfa1-477d-ac97-7c5fea526bbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:58:48.229775 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1a69b0-bfa1-477d-ac97-7c5fea526bbe None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:58:48.230063 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1a69b0-bfa1-477d-ac97-7c5fea526bbe None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:58:48.230343 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1a69b0-bfa1-477d-ac97-7c5fea526bbe None admin] PATH_INFO: `/v3/services` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:58:48.231045 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1a69b0-bfa1-477d-ac97-7c5fea526bbe None admin] RBAC: Adding query filter params (name=placement) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:58:48.231355 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1a69b0-bfa1-477d-ac97-7c5fea526bbe None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:58:48.232716 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1a69b0-bfa1-477d-ac97-7c5fea526bbe None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:58:48.238113 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 655/1291] 10.222.0.46 () {56 vars in 1116 bytes} [Thu May 26 14:58:48 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:58:48.243637 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e8ab8f-7a71-449f-bfac-1b01b35d7d1e None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:58:48.340592 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e8ab8f-7a71-449f-bfac-1b01b35d7d1e None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:58:48.374610 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445c98ff-9c1e-4484-8dbe-3c8ee01e2aac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:58:48.376224 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445c98ff-9c1e-4484-8dbe-3c8ee01e2aac None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:58:48.376607 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445c98ff-9c1e-4484-8dbe-3c8ee01e2aac None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:58:48.376945 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445c98ff-9c1e-4484-8dbe-3c8ee01e2aac None admin] PATH_INFO: `/v3/services` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:58:48.377993 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445c98ff-9c1e-4484-8dbe-3c8ee01e2aac None admin] RBAC: Adding query filter params (type=placement) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:58:48.378377 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445c98ff-9c1e-4484-8dbe-3c8ee01e2aac None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:58:48.380280 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445c98ff-9c1e-4484-8dbe-3c8ee01e2aac None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:58:48.386093 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 637/1292] 10.222.0.46 () {56 vars in 1116 bytes} [Thu May 26 14:58:48 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:58:50.542884 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e163099b-271a-4c76-b874-b6d65de0af9a None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:58:50.543415 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e163099b-271a-4c76-b874-b6d65de0af9a None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:58:50.543415 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e163099b-271a-4c76-b874-b6d65de0af9a None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:58:50.544799 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 656/1293] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 14:58:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 14:58:50.549989 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4279230e-9bfb-4407-b3d0-0982a2b0c619 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:58:50.550361 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4279230e-9bfb-4407-b3d0-0982a2b0c619 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:58:50.550553 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4279230e-9bfb-4407-b3d0-0982a2b0c619 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:58:50.566867 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-4279230e-9bfb-4407-b3d0-0982a2b0c619 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:58:50.568047 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-4279230e-9bfb-4407-b3d0-0982a2b0c619 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:58:50.597359 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:58:50.633539 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 638/1294] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 14:58:50 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 14:58:50.655833 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e163099b-271a-4c76-b874-b6d65de0af9a None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:58:50.686264 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:58:50.755353 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e163099b-271a-4c76-b874-b6d65de0af9a None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:58:50.788257 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2be330d-66fd-4aa4-bf96-9d5ef711305b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:58:50.788905 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2be330d-66fd-4aa4-bf96-9d5ef711305b None admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:58:50.789187 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2be330d-66fd-4aa4-bf96-9d5ef711305b None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:58:50.789540 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2be330d-66fd-4aa4-bf96-9d5ef711305b None admin] PATH_INFO: `/v3/services` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:58:50.790209 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2be330d-66fd-4aa4-bf96-9d5ef711305b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:58:50.791578 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2be330d-66fd-4aa4-bf96-9d5ef711305b None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:58:50.803825 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 657/1295] 10.222.0.46 () {60 vars in 1141 bytes} [Thu May 26 14:58:50 2022] POST /identity/v3/services => generated 246 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 14:58:52.978797 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cce96a2-5ceb-4332-9c28-37aabd28cbb3 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:58:52.979318 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cce96a2-5ceb-4332-9c28-37aabd28cbb3 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:58:52.979318 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cce96a2-5ceb-4332-9c28-37aabd28cbb3 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:58:52.980645 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 639/1296] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 14:58:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 14:58:52.986310 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d11f4e-78d3-4605-98ed-9c51cf2993d1 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:58:52.986701 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d11f4e-78d3-4605-98ed-9c51cf2993d1 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:58:52.986863 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d11f4e-78d3-4605-98ed-9c51cf2993d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:58:53.004098 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-71d11f4e-78d3-4605-98ed-9c51cf2993d1 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:58:53.005264 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-71d11f4e-78d3-4605-98ed-9c51cf2993d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:58:53.075000 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 658/1297] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 14:58:52 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 26 14:58:53.097250 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cce96a2-5ceb-4332-9c28-37aabd28cbb3 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:58:53.194124 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cce96a2-5ceb-4332-9c28-37aabd28cbb3 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:58:53.226603 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93800c62-8693-48bc-9226-1a3a077d7bfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:58:53.227391 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93800c62-8693-48bc-9226-1a3a077d7bfa None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:58:53.227674 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93800c62-8693-48bc-9226-1a3a077d7bfa None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:58:53.227951 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93800c62-8693-48bc-9226-1a3a077d7bfa None admin] PATH_INFO: `/v3/services/placement` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:58:53.228593 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93800c62-8693-48bc-9226-1a3a077d7bfa None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:58:53.230035 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93800c62-8693-48bc-9226-1a3a077d7bfa None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:58:53.242252 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-93800c62-8693-48bc-9226-1a3a077d7bfa None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 26 14:58:53.242252 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:58:53.242252 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 14:58:53.242252 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 14:58:53.242252 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 14:58:53.242252 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 14:58:53.242252 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 14:58:53.242252 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 14:58:53.242252 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 26 14:58:53.242252 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 26 14:58:53.242252 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 14:58:53.242252 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 26 14:58:53.242252 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:58:53.242252 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:58:53.242252 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:58:53.242252 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:58:53.242252 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:58:53.243393 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:58:53.243393 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:58:53.243393 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:58:53.243393 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:58:53.243393 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:58:53.243393 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:58:53.243393 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 14:58:53.243393 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 14:58:53.243393 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 14:58:53.243393 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 14:58:53.243393 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:58:53.243393 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:58:53.243393 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:58:53.243393 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:58:53.243393 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:58:53.243393 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:58:53.244429 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:58:53.244429 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 26 14:58:53.244429 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:58:53.244429 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 26 14:58:53.244429 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:58:53.244429 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:58:53.244429 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:58:53.244429 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 26 14:58:53.244429 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:58:53.244429 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 26 14:58:53.244429 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 14:58:53.244429 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 14:58:53.244429 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 26 14:58:53.244429 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 26 14:58:53.244429 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 640/1298] 10.222.0.46 () {56 vars in 1117 bytes} [Thu May 26 14:58:53 2022] GET /identity/v3/services/placement => generated 90 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:58:53.249240 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d11f4e-78d3-4605-98ed-9c51cf2993d1 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:58:53.267101 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:58:53.345751 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d11f4e-78d3-4605-98ed-9c51cf2993d1 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:58:53.381468 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9141717-7f5d-4b2d-b00e-a8e88067f6e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:58:53.382239 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9141717-7f5d-4b2d-b00e-a8e88067f6e4 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:58:53.382525 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9141717-7f5d-4b2d-b00e-a8e88067f6e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:58:53.382806 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9141717-7f5d-4b2d-b00e-a8e88067f6e4 None admin] PATH_INFO: `/v3/services` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:58:53.383518 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9141717-7f5d-4b2d-b00e-a8e88067f6e4 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:58:53.383829 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9141717-7f5d-4b2d-b00e-a8e88067f6e4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:58:53.385190 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9141717-7f5d-4b2d-b00e-a8e88067f6e4 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:58:53.390306 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 659/1299] 10.222.0.46 () {56 vars in 1116 bytes} [Thu May 26 14:58:53 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:58:53.397145 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93800c62-8693-48bc-9226-1a3a077d7bfa None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:58:53.430175 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:58:53.496042 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93800c62-8693-48bc-9226-1a3a077d7bfa None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:58:53.541071 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807e85be-ce35-4ef5-8709-64d7a5f42435 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:58:53.541758 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807e85be-ce35-4ef5-8709-64d7a5f42435 None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:58:53.542041 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807e85be-ce35-4ef5-8709-64d7a5f42435 None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:58:53.542315 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807e85be-ce35-4ef5-8709-64d7a5f42435 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:58:53.543106 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807e85be-ce35-4ef5-8709-64d7a5f42435 None admin] RBAC: Adding query filter params (interface=public, service_id=bf8c9d36c72442ab8caba56a27276520, region_id=RegionOne) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:58:53.543429 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807e85be-ce35-4ef5-8709-64d7a5f42435 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:58:53.545199 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807e85be-ce35-4ef5-8709-64d7a5f42435 None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:58:53.550592 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 641/1300] 10.222.0.46 () {56 vars in 1251 bytes} [Thu May 26 14:58:53 2022] GET /identity/v3/endpoints?service_id=bf8c9d36c72442ab8caba56a27276520&interface=public®ion_id=RegionOne => generated 194 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:58:53.556257 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9141717-7f5d-4b2d-b00e-a8e88067f6e4 None admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:58:53.637292 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:58:53.655727 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9141717-7f5d-4b2d-b00e-a8e88067f6e4 None admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:58:53.688854 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0eb5eb-7440-4dcc-9c11-dac2ad3170bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:58:53.689535 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0eb5eb-7440-4dcc-9c11-dac2ad3170bc None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:58:53.689825 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0eb5eb-7440-4dcc-9c11-dac2ad3170bc None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:58:53.690107 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0eb5eb-7440-4dcc-9c11-dac2ad3170bc None admin] PATH_INFO: `/v3/services` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:58:53.690782 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0eb5eb-7440-4dcc-9c11-dac2ad3170bc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:58:53.692145 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0eb5eb-7440-4dcc-9c11-dac2ad3170bc None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:58:53.698823 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 660/1301] 10.222.0.46 () {56 vars in 1087 bytes} [Thu May 26 14:58:53 2022] GET /identity/v3/services => generated 2191 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 14:58:55.854398 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a385f701-7008-4dc6-af6c-9caed58ff8a7 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:58:55.854946 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a385f701-7008-4dc6-af6c-9caed58ff8a7 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:58:55.854946 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a385f701-7008-4dc6-af6c-9caed58ff8a7 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:58:55.856297 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 642/1302] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 14:58:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 14:58:55.861908 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dfb4b8-4782-4cf8-856b-6c1946eac7c1 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:58:55.862324 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dfb4b8-4782-4cf8-856b-6c1946eac7c1 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:58:55.862472 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dfb4b8-4782-4cf8-856b-6c1946eac7c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:58:55.879691 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-93dfb4b8-4782-4cf8-856b-6c1946eac7c1 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 14:58:55.880806 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-93dfb4b8-4782-4cf8-856b-6c1946eac7c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 14:58:55.946413 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 661/1303] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 14:58:55 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 26 14:58:55.968196 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a385f701-7008-4dc6-af6c-9caed58ff8a7 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:58:56.058814 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:58:56.066239 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a385f701-7008-4dc6-af6c-9caed58ff8a7 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:58:56.099010 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7c4e1d-24b2-4a88-9b24-89c41527095b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:58:56.099789 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7c4e1d-24b2-4a88-9b24-89c41527095b None admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:58:56.100068 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7c4e1d-24b2-4a88-9b24-89c41527095b None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:58:56.100339 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7c4e1d-24b2-4a88-9b24-89c41527095b None admin] PATH_INFO: `/v3/services/placement` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:58:56.100984 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7c4e1d-24b2-4a88-9b24-89c41527095b None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:58:56.102476 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7c4e1d-24b2-4a88-9b24-89c41527095b None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:58:56.114569 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application [None req-0a7c4e1d-24b2-4a88-9b24-89c41527095b None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 26 14:58:56.114569 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:58:56.114569 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 26 14:58:56.114569 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 26 14:58:56.114569 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 26 14:58:56.114569 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 26 14:58:56.114569 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 26 14:58:56.114569 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 14:58:56.114569 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 26 14:58:56.114569 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 26 14:58:56.114569 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 26 14:58:56.114569 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 26 14:58:56.114569 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application Traceback (most recent call last): May 26 14:58:56.114569 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 26 14:58:56.114569 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 26 14:58:56.114569 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 26 14:58:56.114569 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 26 14:58:56.115761 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 26 14:58:56.115761 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 26 14:58:56.115761 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 26 14:58:56.115761 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 26 14:58:56.115761 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 26 14:58:56.115761 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 26 14:58:56.115761 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 26 14:58:56.115761 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._get_service(service_id) May 26 14:58:56.115761 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 26 14:58:56.115761 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 26 14:58:56.115761 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 26 14:58:56.115761 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 26 14:58:56.115761 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 26 14:58:56.115761 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 26 14:58:56.115761 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 26 14:58:56.115761 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.get_or_create( May 26 14:58:56.116646 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 26 14:58:56.116646 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application with Lock( May 26 14:58:56.116646 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 26 14:58:56.116646 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self._enter() May 26 14:58:56.116646 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 26 14:58:56.116646 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 26 14:58:56.116646 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 26 14:58:56.116646 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application return self.creator() May 26 14:58:56.116646 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 26 14:58:56.116646 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application created_value = creator( May 26 14:58:56.116646 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 26 14:58:56.116646 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 26 14:58:56.116646 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 26 14:58:56.116646 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: ERROR keystone.server.flask.application May 26 14:58:56.116646 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 643/1304] 10.222.0.46 () {56 vars in 1117 bytes} [Thu May 26 14:58:55 2022] GET /identity/v3/services/placement => generated 90 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 26 14:58:56.121640 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93dfb4b8-4782-4cf8-856b-6c1946eac7c1 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:58:56.195190 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:58:56.216949 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93dfb4b8-4782-4cf8-856b-6c1946eac7c1 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:58:56.250469 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020b7c3d-e64a-454f-ad44-c8ce51d308d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:58:56.251383 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020b7c3d-e64a-454f-ad44-c8ce51d308d7 None admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:58:56.251668 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020b7c3d-e64a-454f-ad44-c8ce51d308d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:58:56.251947 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020b7c3d-e64a-454f-ad44-c8ce51d308d7 None admin] PATH_INFO: `/v3/services` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:58:56.252647 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020b7c3d-e64a-454f-ad44-c8ce51d308d7 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 14:58:56.252958 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020b7c3d-e64a-454f-ad44-c8ce51d308d7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:58:56.254341 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020b7c3d-e64a-454f-ad44-c8ce51d308d7 None admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:58:56.259358 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 662/1305] 10.222.0.46 () {56 vars in 1116 bytes} [Thu May 26 14:58:56 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 14:58:56.264976 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7c4e1d-24b2-4a88-9b24-89c41527095b None admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 14:58:56.360721 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7c4e1d-24b2-4a88-9b24-89c41527095b None admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 14:58:56.397386 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f3c21e1-4483-4b1d-8c00-820a4300d33c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 14:58:56.398070 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3c21e1-4483-4b1d-8c00-820a4300d33c None admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 14:58:56.398356 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3c21e1-4483-4b1d-8c00-820a4300d33c None admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 14:58:56.398635 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f3c21e1-4483-4b1d-8c00-820a4300d33c None admin] PATH_INFO: `/v3/endpoints` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 14:58:56.399362 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3c21e1-4483-4b1d-8c00-820a4300d33c None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 14:58:56.400723 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f3c21e1-4483-4b1d-8c00-820a4300d33c None admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 14:58:56.419391 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 644/1306] 10.222.0.46 () {60 vars in 1144 bytes} [Thu May 26 14:58:56 2022] POST /identity/v3/endpoints => generated 331 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 14:59:06.256433 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=30 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 14:59:06.415162 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:00:01.410720 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b68b07-7774-44e5-b108-09c699aded4e None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:00:01.411264 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b68b07-7774-44e5-b108-09c699aded4e None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:00:01.411264 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b68b07-7774-44e5-b108-09c699aded4e None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:00:01.416601 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 663/1307] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 15:00:01 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 15:00:01.418106 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33f49ab-558c-4b8a-a7c0-b8fdff8af0ad None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:00:01.418539 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33f49ab-558c-4b8a-a7c0-b8fdff8af0ad None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:00:01.418674 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33f49ab-558c-4b8a-a7c0-b8fdff8af0ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:00:01.515213 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-b33f49ab-558c-4b8a-a7c0-b8fdff8af0ad None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:00:01.517062 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-b33f49ab-558c-4b8a-a7c0-b8fdff8af0ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:00:01.677622 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 645/1308] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 15:00:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:00:01.734668 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfdfc99-05e3-4900-a5a4-40720113f0e0 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:00:01.734986 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfdfc99-05e3-4900-a5a4-40720113f0e0 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:00:01.735382 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bfdfc99-05e3-4900-a5a4-40720113f0e0 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:00:01.738085 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 664/1309] 10.222.0.46 () {54 vars in 964 bytes} [Thu May 26 15:00:01 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 15:00:01.746528 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef62784-9852-4fa4-a4b5-d9cffc818fa6 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:00:01.746831 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef62784-9852-4fa4-a4b5-d9cffc818fa6 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:00:01.747381 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef62784-9852-4fa4-a4b5-d9cffc818fa6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:00:01.850304 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-aef62784-9852-4fa4-a4b5-d9cffc818fa6 None None] MFA Rules not processed for user `0156a2f6f3794f479e76b85bdbbe5c26`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:00:01.851538 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-aef62784-9852-4fa4-a4b5-d9cffc818fa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:00:01.873220 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:00:01.932802 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 646/1310] 10.222.0.46 () {58 vars in 1061 bytes} [Thu May 26 15:00:01 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:00:01.941847 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfdfc99-05e3-4900-a5a4-40720113f0e0 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:00:01.943801 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-1bfdfc99-05e3-4900-a5a4-40720113f0e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:00:02.073381 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bfdfc99-05e3-4900-a5a4-40720113f0e0 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:00:02.158399 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79daa23b-3475-4292-8664-974df6cf5593 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0156a2f6f3794f479e76b85bdbbe5c26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:00:02.159058 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79daa23b-3475-4292-8664-974df6cf5593 service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:00:02.159339 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79daa23b-3475-4292-8664-974df6cf5593 service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:00:02.159611 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79daa23b-3475-4292-8664-974df6cf5593 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:00:02.161350 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-79daa23b-3475-4292-8664-974df6cf5593 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:00:02.220581 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:00:02.290975 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79daa23b-3475-4292-8664-974df6cf5593 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:00:02.292433 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79daa23b-3475-4292-8664-974df6cf5593 service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:00:02.313496 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 665/1311] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:00:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 15:00:05.094413 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acc6950-9e25-456c-a6a7-43f42c370475 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:00:05.094939 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acc6950-9e25-456c-a6a7-43f42c370475 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:00:05.094939 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acc6950-9e25-456c-a6a7-43f42c370475 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:00:05.100017 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 647/1312] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 15:00:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 15:00:05.102313 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc3dd89-686e-4939-b3d4-122318f9475a None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:00:05.102776 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc3dd89-686e-4939-b3d4-122318f9475a None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:00:05.102882 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc3dd89-686e-4939-b3d4-122318f9475a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:00:05.196007 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-7dc3dd89-686e-4939-b3d4-122318f9475a None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:00:05.197237 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-7dc3dd89-686e-4939-b3d4-122318f9475a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:00:05.328522 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 666/1313] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 15:00:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:00:05.387684 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7683f81-a20c-47b1-be8c-c5e1367d3bf0 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:00:05.388575 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7683f81-a20c-47b1-be8c-c5e1367d3bf0 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:00:05.388859 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7683f81-a20c-47b1-be8c-c5e1367d3bf0 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:00:05.390472 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 648/1314] 10.222.0.46 () {54 vars in 964 bytes} [Thu May 26 15:00:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 15:00:05.397227 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c218de26-0c10-4ccc-acde-ab4c35bab818 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:00:05.397684 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c218de26-0c10-4ccc-acde-ab4c35bab818 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:00:05.398058 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c218de26-0c10-4ccc-acde-ab4c35bab818 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:00:05.409417 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:00:05.479611 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-c218de26-0c10-4ccc-acde-ab4c35bab818 None None] MFA Rules not processed for user `0156a2f6f3794f479e76b85bdbbe5c26`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:00:05.481031 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-c218de26-0c10-4ccc-acde-ab4c35bab818 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:00:05.577486 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 667/1315] 10.222.0.46 () {58 vars in 1061 bytes} [Thu May 26 15:00:05 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:00:05.587025 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7683f81-a20c-47b1-be8c-c5e1367d3bf0 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:00:05.588587 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-a7683f81-a20c-47b1-be8c-c5e1367d3bf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:00:05.717181 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7683f81-a20c-47b1-be8c-c5e1367d3bf0 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:00:05.807452 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:00:05.812157 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4ca769-07f9-419c-aa1f-b9dc2f1a6f6c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0156a2f6f3794f479e76b85bdbbe5c26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:00:05.813351 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4ca769-07f9-419c-aa1f-b9dc2f1a6f6c service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:00:05.814053 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4ca769-07f9-419c-aa1f-b9dc2f1a6f6c service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:00:05.814725 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4ca769-07f9-419c-aa1f-b9dc2f1a6f6c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:00:05.816740 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-2c4ca769-07f9-419c-aa1f-b9dc2f1a6f6c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:00:05.974566 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4ca769-07f9-419c-aa1f-b9dc2f1a6f6c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:00:05.976401 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4ca769-07f9-419c-aa1f-b9dc2f1a6f6c service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:00:06.001369 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 649/1316] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:00:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 15:00:08.904282 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5479ec74-5b59-452d-9ac2-f98ec9d5d3f6 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:00:08.904850 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5479ec74-5b59-452d-9ac2-f98ec9d5d3f6 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:00:08.904850 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5479ec74-5b59-452d-9ac2-f98ec9d5d3f6 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:00:08.906226 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 668/1317] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 15:00:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 15:00:08.911778 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b2da2e-b98a-4220-b608-dc1d98fe40cf None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:00:08.912211 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b2da2e-b98a-4220-b608-dc1d98fe40cf None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:00:08.912335 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b2da2e-b98a-4220-b608-dc1d98fe40cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:00:08.976040 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-04b2da2e-b98a-4220-b608-dc1d98fe40cf None None] MFA Rules not processed for user `70e4c37945d34ae18f6ac64fb237b230`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:00:08.977602 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-04b2da2e-b98a-4220-b608-dc1d98fe40cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:00:09.034003 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:00:09.079894 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 650/1318] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 15:00:08 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:00:09.121104 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5479ec74-5b59-452d-9ac2-f98ec9d5d3f6 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:00:09.122889 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-5479ec74-5b59-452d-9ac2-f98ec9d5d3f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:00:09.227721 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5479ec74-5b59-452d-9ac2-f98ec9d5d3f6 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:00:09.230745 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:00:09.304119 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e662a7b9-261c-4866-a477-a4a83bacd738 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0156a2f6f3794f479e76b85bdbbe5c26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:00:09.304767 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e662a7b9-261c-4866-a477-a4a83bacd738 service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:00:09.305050 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e662a7b9-261c-4866-a477-a4a83bacd738 service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:00:09.305324 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e662a7b9-261c-4866-a477-a4a83bacd738 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:00:09.307036 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-e662a7b9-261c-4866-a477-a4a83bacd738 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:00:09.421673 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e662a7b9-261c-4866-a477-a4a83bacd738 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:00:09.423119 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e662a7b9-261c-4866-a477-a4a83bacd738 service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:00:09.442049 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 669/1319] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:00:09 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 15:00:12.294180 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42646dcc-233a-411a-b896-a7a80d861d96 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:00:12.294730 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42646dcc-233a-411a-b896-a7a80d861d96 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:00:12.294730 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42646dcc-233a-411a-b896-a7a80d861d96 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:00:12.296173 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 651/1320] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 15:00:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 15:00:12.301563 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98da187-34cb-4d84-bbd3-fb07781efe71 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:00:12.302028 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98da187-34cb-4d84-bbd3-fb07781efe71 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:00:12.302125 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98da187-34cb-4d84-bbd3-fb07781efe71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:00:12.345056 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:00:12.374769 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-e98da187-34cb-4d84-bbd3-fb07781efe71 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:00:12.376384 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-e98da187-34cb-4d84-bbd3-fb07781efe71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:00:12.490983 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 670/1321] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 15:00:12 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:00:12.533016 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42646dcc-233a-411a-b896-a7a80d861d96 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:00:12.534594 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-42646dcc-233a-411a-b896-a7a80d861d96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:00:12.650142 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42646dcc-233a-411a-b896-a7a80d861d96 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:00:12.746871 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76da8f0a-395d-47db-b4af-73fa69117918 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0156a2f6f3794f479e76b85bdbbe5c26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:00:12.749904 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76da8f0a-395d-47db-b4af-73fa69117918 service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:00:12.750186 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76da8f0a-395d-47db-b4af-73fa69117918 service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:00:12.750715 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76da8f0a-395d-47db-b4af-73fa69117918 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:00:12.752522 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-76da8f0a-395d-47db-b4af-73fa69117918 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:00:12.760486 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:00:12.869613 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76da8f0a-395d-47db-b4af-73fa69117918 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:00:12.871045 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76da8f0a-395d-47db-b4af-73fa69117918 service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:00:12.890938 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 652/1322] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:00:12 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 15:00:16.020745 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e561fe1-01d3-4018-816c-0cc534d83a70 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:00:16.021327 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e561fe1-01d3-4018-816c-0cc534d83a70 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:00:16.021327 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e561fe1-01d3-4018-816c-0cc534d83a70 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:00:16.023455 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 671/1323] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 15:00:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 15:00:16.031580 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd6e70a-c64c-406a-8d23-1a953a68ddb8 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:00:16.032112 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd6e70a-c64c-406a-8d23-1a953a68ddb8 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:00:16.032225 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd6e70a-c64c-406a-8d23-1a953a68ddb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:00:16.134558 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-6cd6e70a-c64c-406a-8d23-1a953a68ddb8 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:00:16.136627 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-6cd6e70a-c64c-406a-8d23-1a953a68ddb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:00:16.235091 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:00:16.287718 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 653/1324] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 15:00:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:00:16.327375 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e561fe1-01d3-4018-816c-0cc534d83a70 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:00:16.329586 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-3e561fe1-01d3-4018-816c-0cc534d83a70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:00:16.441590 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e561fe1-01d3-4018-816c-0cc534d83a70 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:00:16.454263 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:00:16.522483 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d060708-1531-4802-ba80-cc91153ace92 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0156a2f6f3794f479e76b85bdbbe5c26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:00:16.523135 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d060708-1531-4802-ba80-cc91153ace92 service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:00:16.523417 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d060708-1531-4802-ba80-cc91153ace92 service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:00:16.523690 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d060708-1531-4802-ba80-cc91153ace92 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:00:16.525333 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-8d060708-1531-4802-ba80-cc91153ace92 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:00:16.643758 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d060708-1531-4802-ba80-cc91153ace92 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:00:16.645251 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d060708-1531-4802-ba80-cc91153ace92 service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:00:16.665679 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 672/1325] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:00:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 15:00:18.978878 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c17ce13-b86f-4226-be44-bb364529aeff None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:00:18.979671 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c17ce13-b86f-4226-be44-bb364529aeff None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:00:18.979671 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c17ce13-b86f-4226-be44-bb364529aeff None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:00:18.981152 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 654/1326] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 15:00:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 15:00:18.987650 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02054eec-5437-4e1a-9a30-c138791b0dce None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:00:18.988187 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02054eec-5437-4e1a-9a30-c138791b0dce None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:00:18.988337 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02054eec-5437-4e1a-9a30-c138791b0dce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:00:19.039538 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:00:19.063982 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-02054eec-5437-4e1a-9a30-c138791b0dce None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:00:19.065225 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-02054eec-5437-4e1a-9a30-c138791b0dce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:00:19.175970 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 673/1327] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 15:00:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:00:19.211721 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c17ce13-b86f-4226-be44-bb364529aeff None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:00:19.213266 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-4c17ce13-b86f-4226-be44-bb364529aeff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:00:19.335449 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c17ce13-b86f-4226-be44-bb364529aeff None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:00:19.414957 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595f9b8d-4212-44d0-899b-ee449602e3d6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0156a2f6f3794f479e76b85bdbbe5c26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:00:19.415673 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595f9b8d-4212-44d0-899b-ee449602e3d6 service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:00:19.415964 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595f9b8d-4212-44d0-899b-ee449602e3d6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:00:19.416244 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595f9b8d-4212-44d0-899b-ee449602e3d6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:00:19.417956 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-595f9b8d-4212-44d0-899b-ee449602e3d6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:00:19.435707 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:00:19.543245 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595f9b8d-4212-44d0-899b-ee449602e3d6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:00:19.545275 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595f9b8d-4212-44d0-899b-ee449602e3d6 service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:00:19.563778 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 655/1328] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:00:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 15:00:21.876993 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e7e25d-e470-4bee-acfe-b01e7bf137a5 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:00:21.877603 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e7e25d-e470-4bee-acfe-b01e7bf137a5 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:00:21.877603 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e7e25d-e470-4bee-acfe-b01e7bf137a5 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:00:21.878961 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 674/1329] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 15:00:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 15:00:21.884556 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030dbf03-4e44-4d02-a4d9-b75d2a6b0a8f None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:00:21.885039 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030dbf03-4e44-4d02-a4d9-b75d2a6b0a8f None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:00:21.885149 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030dbf03-4e44-4d02-a4d9-b75d2a6b0a8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:00:21.953937 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-030dbf03-4e44-4d02-a4d9-b75d2a6b0a8f None None] MFA Rules not processed for user `70e4c37945d34ae18f6ac64fb237b230`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:00:21.955161 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-030dbf03-4e44-4d02-a4d9-b75d2a6b0a8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:00:22.023081 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:00:22.062124 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 656/1330] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 15:00:21 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:00:22.101353 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e7e25d-e470-4bee-acfe-b01e7bf137a5 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:00:22.103052 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-41e7e25d-e470-4bee-acfe-b01e7bf137a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:00:22.233523 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e7e25d-e470-4bee-acfe-b01e7bf137a5 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:00:22.260216 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:00:22.333922 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc00914-b765-4c6b-9d03-80f3768da023 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0156a2f6f3794f479e76b85bdbbe5c26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:00:22.335038 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc00914-b765-4c6b-9d03-80f3768da023 service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:00:22.335494 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc00914-b765-4c6b-9d03-80f3768da023 service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:00:22.335902 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc00914-b765-4c6b-9d03-80f3768da023 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:00:22.338846 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-cdc00914-b765-4c6b-9d03-80f3768da023 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:00:22.469946 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc00914-b765-4c6b-9d03-80f3768da023 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:00:22.471432 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc00914-b765-4c6b-9d03-80f3768da023 service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:00:22.490101 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 675/1331] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:00:22 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 15:00:25.156124 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc458c3e-0a8f-4949-9bbc-6a455be3f59d None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:00:25.156643 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc458c3e-0a8f-4949-9bbc-6a455be3f59d None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:00:25.156767 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc458c3e-0a8f-4949-9bbc-6a455be3f59d None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:00:25.158161 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 657/1332] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 15:00:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 15:00:25.163480 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90408f2a-42cf-4a6c-a248-198bc90ddb4f None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:00:25.163893 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90408f2a-42cf-4a6c-a248-198bc90ddb4f None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:00:25.164040 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90408f2a-42cf-4a6c-a248-198bc90ddb4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:00:25.227295 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:00:25.230293 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-90408f2a-42cf-4a6c-a248-198bc90ddb4f None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:00:25.232176 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-90408f2a-42cf-4a6c-a248-198bc90ddb4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:00:25.341326 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 676/1333] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 15:00:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:00:25.380990 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc458c3e-0a8f-4949-9bbc-6a455be3f59d None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:00:25.382706 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-cc458c3e-0a8f-4949-9bbc-6a455be3f59d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:00:25.492858 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc458c3e-0a8f-4949-9bbc-6a455be3f59d None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:00:25.579406 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08f67a3-4375-474e-adf0-73c4e443186a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0156a2f6f3794f479e76b85bdbbe5c26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:00:25.580070 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08f67a3-4375-474e-adf0-73c4e443186a service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:00:25.580354 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08f67a3-4375-474e-adf0-73c4e443186a service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:00:25.580632 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08f67a3-4375-474e-adf0-73c4e443186a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:00:25.582358 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-b08f67a3-4375-474e-adf0-73c4e443186a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:00:25.613124 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:00:25.708958 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08f67a3-4375-474e-adf0-73c4e443186a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:00:25.710461 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08f67a3-4375-474e-adf0-73c4e443186a service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:00:25.732922 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 658/1334] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:00:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 15:00:28.013048 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88186b3a-4299-45a3-af0f-ac26a6dd1617 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:00:28.013594 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88186b3a-4299-45a3-af0f-ac26a6dd1617 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:00:28.013594 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88186b3a-4299-45a3-af0f-ac26a6dd1617 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:00:28.014919 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 677/1335] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 15:00:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 15:00:28.020096 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811feb50-79f5-4c5a-ace2-c99d451eb268 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:00:28.020496 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811feb50-79f5-4c5a-ace2-c99d451eb268 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:00:28.020651 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811feb50-79f5-4c5a-ace2-c99d451eb268 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:00:28.090024 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-811feb50-79f5-4c5a-ace2-c99d451eb268 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:00:28.091996 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-811feb50-79f5-4c5a-ace2-c99d451eb268 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:00:28.168522 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:00:28.200814 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 659/1336] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 15:00:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:00:28.240353 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88186b3a-4299-45a3-af0f-ac26a6dd1617 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:00:28.241937 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-88186b3a-4299-45a3-af0f-ac26a6dd1617 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:00:28.351536 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88186b3a-4299-45a3-af0f-ac26a6dd1617 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:00:28.381229 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:00:28.431101 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd826e0d-d548-426c-a37e-6e971b7a08c2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0156a2f6f3794f479e76b85bdbbe5c26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:00:28.431860 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd826e0d-d548-426c-a37e-6e971b7a08c2 service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:00:28.432207 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd826e0d-d548-426c-a37e-6e971b7a08c2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:00:28.432521 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd826e0d-d548-426c-a37e-6e971b7a08c2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:00:28.434601 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-bd826e0d-d548-426c-a37e-6e971b7a08c2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:00:28.548619 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd826e0d-d548-426c-a37e-6e971b7a08c2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:00:28.550107 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd826e0d-d548-426c-a37e-6e971b7a08c2 service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:00:28.568170 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 678/1337] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:00:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 15:00:31.138789 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065fcecc-8e94-468e-a502-ecf2748ac824 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:00:31.139585 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065fcecc-8e94-468e-a502-ecf2748ac824 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:00:31.139585 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065fcecc-8e94-468e-a502-ecf2748ac824 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:00:31.140729 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 660/1338] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 15:00:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 15:00:31.146171 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d03133-deef-4a96-b745-68c068a61c9c None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:00:31.146695 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d03133-deef-4a96-b745-68c068a61c9c None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:00:31.146794 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d03133-deef-4a96-b745-68c068a61c9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:00:31.212744 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-41d03133-deef-4a96-b745-68c068a61c9c None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:00:31.213992 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-41d03133-deef-4a96-b745-68c068a61c9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:00:31.227060 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:00:31.315394 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 679/1339] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 15:00:31 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 15:00:31.351102 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065fcecc-8e94-468e-a502-ecf2748ac824 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:00:31.352687 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-065fcecc-8e94-468e-a502-ecf2748ac824 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:00:31.464426 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065fcecc-8e94-468e-a502-ecf2748ac824 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:00:31.563351 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ba81a5-4761-45f2-9ae1-ab0faae2a126 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0156a2f6f3794f479e76b85bdbbe5c26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:00:31.564010 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ba81a5-4761-45f2-9ae1-ab0faae2a126 service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:00:31.564296 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ba81a5-4761-45f2-9ae1-ab0faae2a126 service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:00:31.564575 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ba81a5-4761-45f2-9ae1-ab0faae2a126 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:00:31.566678 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-84ba81a5-4761-45f2-9ae1-ab0faae2a126 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:00:31.604771 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:00:31.693620 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ba81a5-4761-45f2-9ae1-ab0faae2a126 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:00:31.695903 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ba81a5-4761-45f2-9ae1-ab0faae2a126 service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:00:31.727584 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 661/1340] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:00:31 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 15:00:36.170273 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868a5a94-f3cb-4a76-a1fc-ccd574c689a9 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:00:36.170872 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868a5a94-f3cb-4a76-a1fc-ccd574c689a9 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:00:36.170872 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868a5a94-f3cb-4a76-a1fc-ccd574c689a9 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:00:36.172134 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 680/1341] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 15:00:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 15:00:36.177528 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a887fe97-3153-4f38-862c-2b9fca074891 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:00:36.177924 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a887fe97-3153-4f38-862c-2b9fca074891 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:00:36.178098 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a887fe97-3153-4f38-862c-2b9fca074891 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:00:36.226403 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-a887fe97-3153-4f38-862c-2b9fca074891 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:00:36.227518 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-a887fe97-3153-4f38-862c-2b9fca074891 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:00:36.309666 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 662/1342] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 15:00:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:00:36.348846 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868a5a94-f3cb-4a76-a1fc-ccd574c689a9 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:00:36.351191 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-868a5a94-f3cb-4a76-a1fc-ccd574c689a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:00:36.391639 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868a5a94-f3cb-4a76-a1fc-ccd574c689a9 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:00:36.395843 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d882234-979c-4b56-bc86-21b6e1623c76 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0156a2f6f3794f479e76b85bdbbe5c26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:00:36.396566 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d882234-979c-4b56-bc86-21b6e1623c76 service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:00:36.396788 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d882234-979c-4b56-bc86-21b6e1623c76 service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:00:36.397066 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d882234-979c-4b56-bc86-21b6e1623c76 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:00:36.407959 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d882234-979c-4b56-bc86-21b6e1623c76 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:00:36.409418 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d882234-979c-4b56-bc86-21b6e1623c76 service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:00:36.419483 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 681/1343] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:00:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 15:00:39.304545 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2500be-7d7a-4c16-8105-a167dbd4c02e None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:00:39.305105 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2500be-7d7a-4c16-8105-a167dbd4c02e None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:00:39.305105 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2500be-7d7a-4c16-8105-a167dbd4c02e None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:00:39.306523 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 663/1344] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 15:00:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 15:00:39.312642 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c1706f-1e45-4c99-a155-2c5dd4837898 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:00:39.313071 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c1706f-1e45-4c99-a155-2c5dd4837898 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:00:39.313211 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c1706f-1e45-4c99-a155-2c5dd4837898 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:00:39.341241 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-99c1706f-1e45-4c99-a155-2c5dd4837898 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:00:39.342490 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-99c1706f-1e45-4c99-a155-2c5dd4837898 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:00:39.350353 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 682/1345] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 15:00:39 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:00:39.389362 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2500be-7d7a-4c16-8105-a167dbd4c02e None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:00:39.402078 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2500be-7d7a-4c16-8105-a167dbd4c02e None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:00:39.406226 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7e0ee1-423b-44c7-97ed-680547aa0db7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0156a2f6f3794f479e76b85bdbbe5c26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:00:39.406931 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7e0ee1-423b-44c7-97ed-680547aa0db7 service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:00:39.407289 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7e0ee1-423b-44c7-97ed-680547aa0db7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:00:39.407700 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7e0ee1-423b-44c7-97ed-680547aa0db7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:00:39.418168 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7e0ee1-423b-44c7-97ed-680547aa0db7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:00:39.420233 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7e0ee1-423b-44c7-97ed-680547aa0db7 service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:00:39.430710 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 664/1346] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:00:39 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 15:00:43.704205 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbfe8b0-60b9-42cf-8ce1-9e11de01bce7 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:00:43.704773 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbfe8b0-60b9-42cf-8ce1-9e11de01bce7 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:00:43.704773 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbfe8b0-60b9-42cf-8ce1-9e11de01bce7 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:00:43.709899 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 683/1347] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 15:00:43 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 15:00:43.713209 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad6ddc9-cba5-4e2a-8174-42fc310bd593 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:00:43.713749 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad6ddc9-cba5-4e2a-8174-42fc310bd593 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:00:43.713860 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad6ddc9-cba5-4e2a-8174-42fc310bd593 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:00:43.737653 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-8ad6ddc9-cba5-4e2a-8174-42fc310bd593 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:00:43.738891 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-8ad6ddc9-cba5-4e2a-8174-42fc310bd593 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:00:43.746869 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 665/1348] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 15:00:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:00:43.785441 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbfe8b0-60b9-42cf-8ce1-9e11de01bce7 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:00:43.796940 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbfe8b0-60b9-42cf-8ce1-9e11de01bce7 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:00:43.800836 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059f19d4-c3fe-4388-bde5-44d661af65cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0156a2f6f3794f479e76b85bdbbe5c26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:00:43.801520 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059f19d4-c3fe-4388-bde5-44d661af65cf service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:00:43.801810 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059f19d4-c3fe-4388-bde5-44d661af65cf service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:00:43.802091 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059f19d4-c3fe-4388-bde5-44d661af65cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:00:43.813109 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059f19d4-c3fe-4388-bde5-44d661af65cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:00:43.814610 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-059f19d4-c3fe-4388-bde5-44d661af65cf service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:00:43.825024 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 684/1349] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:00:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 15:00:49.487708 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845d00ca-2586-4f34-bc51-f1053968bdc2 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:00:49.488751 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845d00ca-2586-4f34-bc51-f1053968bdc2 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:00:49.489243 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845d00ca-2586-4f34-bc51-f1053968bdc2 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:00:49.491222 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 666/1350] 10.222.0.46 () {54 vars in 921 bytes} [Thu May 26 15:00:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 15:00:49.497086 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d320dde6-0603-469f-b67b-eb381c5b4918 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:00:49.498076 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d320dde6-0603-469f-b67b-eb381c5b4918 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:00:49.498580 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d320dde6-0603-469f-b67b-eb381c5b4918 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:00:49.611453 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-d320dde6-0603-469f-b67b-eb381c5b4918 None None] MFA Rules not processed for user `bf55a00e3b084aa3b22c25a90e6a773c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:00:49.623777 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-d320dde6-0603-469f-b67b-eb381c5b4918 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:00:49.658939 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:00:49.772013 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 685/1351] 10.222.0.46 () {58 vars in 1018 bytes} [Thu May 26 15:00:49 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 276 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:00:53.730935 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=93 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:00:53.956231 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bda2627-9f6c-4bf8-aacd-329598884285 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:00:53.956231 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bda2627-9f6c-4bf8-aacd-329598884285 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:00:53.956231 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bda2627-9f6c-4bf8-aacd-329598884285 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:00:53.957624 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 667/1352] 10.222.0.46 () {54 vars in 921 bytes} [Thu May 26 15:00:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 15:00:53.962800 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893e8cd3-ce49-4a92-98b4-20088fa1f7b4 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:00:53.963543 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893e8cd3-ce49-4a92-98b4-20088fa1f7b4 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:00:53.963543 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893e8cd3-ce49-4a92-98b4-20088fa1f7b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:00:53.982102 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-893e8cd3-ce49-4a92-98b4-20088fa1f7b4 None None] MFA Rules not processed for user `bf55a00e3b084aa3b22c25a90e6a773c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:00:53.983307 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-893e8cd3-ce49-4a92-98b4-20088fa1f7b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:00:53.995073 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 686/1353] 10.222.0.46 () {58 vars in 1018 bytes} [Thu May 26 15:00:53 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:00:55.690122 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17ecb82-1fb3-4d6a-b235-8cb7635d6802 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:00:55.690643 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17ecb82-1fb3-4d6a-b235-8cb7635d6802 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:00:55.690736 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17ecb82-1fb3-4d6a-b235-8cb7635d6802 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:00:55.692051 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 668/1354] 10.222.0.46 () {54 vars in 921 bytes} [Thu May 26 15:00:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 15:00:55.699143 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cbb657-f0a5-41dd-822a-0128fd03a367 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:00:55.700208 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cbb657-f0a5-41dd-822a-0128fd03a367 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:00:55.700303 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cbb657-f0a5-41dd-822a-0128fd03a367 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:00:55.718765 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-69cbb657-f0a5-41dd-822a-0128fd03a367 None None] MFA Rules not processed for user `bf55a00e3b084aa3b22c25a90e6a773c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:00:55.719970 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-69cbb657-f0a5-41dd-822a-0128fd03a367 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:00:55.729361 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 687/1355] 10.222.0.46 () {58 vars in 1018 bytes} [Thu May 26 15:00:55 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:00:59.705081 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f280deb6-6efc-412b-b10e-b2ccfb357d96 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:00:59.705864 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f280deb6-6efc-412b-b10e-b2ccfb357d96 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:00:59.705864 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f280deb6-6efc-412b-b10e-b2ccfb357d96 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:00:59.707193 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 669/1356] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 15:00:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 15:00:59.715402 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507068a7-ffa4-4b55-a629-db4746aa8c99 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:00:59.715948 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507068a7-ffa4-4b55-a629-db4746aa8c99 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:00:59.715948 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507068a7-ffa4-4b55-a629-db4746aa8c99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:00:59.736332 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-507068a7-ffa4-4b55-a629-db4746aa8c99 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:00:59.737872 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-507068a7-ffa4-4b55-a629-db4746aa8c99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:00:59.747265 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 688/1357] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 15:00:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:00:59.803085 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3056a4d-3d2e-4faa-9b51-bf6b2b7e4c07 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:00:59.804113 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3056a4d-3d2e-4faa-9b51-bf6b2b7e4c07 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:00:59.804749 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3056a4d-3d2e-4faa-9b51-bf6b2b7e4c07 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:00:59.807318 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 670/1358] 10.222.0.46 () {54 vars in 961 bytes} [Thu May 26 15:00:59 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 15:00:59.826446 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddb355f-dead-4044-8bc1-515146b79e24 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:00:59.827250 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddb355f-dead-4044-8bc1-515146b79e24 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:00:59.827819 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddb355f-dead-4044-8bc1-515146b79e24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:00:59.883676 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-4ddb355f-dead-4044-8bc1-515146b79e24 None None] MFA Rules not processed for user `d6639353c70449d1b3d10cd6487c3b00`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:00:59.885605 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-4ddb355f-dead-4044-8bc1-515146b79e24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:00:59.989317 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 689/1359] 10.222.0.46 () {58 vars in 1058 bytes} [Thu May 26 15:00:59 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 15:00:59.997780 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3056a4d-3d2e-4faa-9b51-bf6b2b7e4c07 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:01:00.018658 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3056a4d-3d2e-4faa-9b51-bf6b2b7e4c07 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:01:00.025934 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f2e109-3fbb-4c46-bbd5-dc3c73e248c8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6639353c70449d1b3d10cd6487c3b00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:01:00.026906 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f2e109-3fbb-4c46-bbd5-dc3c73e248c8 service nova] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:01:00.028205 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f2e109-3fbb-4c46-bbd5-dc3c73e248c8 service nova] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:01:00.029160 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f2e109-3fbb-4c46-bbd5-dc3c73e248c8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:01:00.044665 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f2e109-3fbb-4c46-bbd5-dc3c73e248c8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:01:00.047831 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f2e109-3fbb-4c46-bbd5-dc3c73e248c8 service nova] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:01:00.059884 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 671/1360] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:00:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 15:01:02.550356 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3817a96f-4b7c-4c8c-8662-fe53b281aa1a None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:01:02.550923 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3817a96f-4b7c-4c8c-8662-fe53b281aa1a None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:01:02.550923 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3817a96f-4b7c-4c8c-8662-fe53b281aa1a None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:01:02.552454 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 690/1361] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 15:01:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 15:01:02.557956 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b7d6ca-af39-4a8d-b777-ef559241f05a None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:01:02.558473 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b7d6ca-af39-4a8d-b777-ef559241f05a None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:01:02.558567 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b7d6ca-af39-4a8d-b777-ef559241f05a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:01:02.577543 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-05b7d6ca-af39-4a8d-b777-ef559241f05a None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:01:02.579062 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-05b7d6ca-af39-4a8d-b777-ef559241f05a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:01:02.589879 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 672/1362] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 15:01:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:01:02.635915 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3817a96f-4b7c-4c8c-8662-fe53b281aa1a None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:01:02.653605 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3817a96f-4b7c-4c8c-8662-fe53b281aa1a None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:01:02.659446 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc903d0-b009-49b7-9ab5-e195ea2b1224 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6639353c70449d1b3d10cd6487c3b00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:01:02.660191 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc903d0-b009-49b7-9ab5-e195ea2b1224 service nova] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:01:02.660544 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc903d0-b009-49b7-9ab5-e195ea2b1224 service nova] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:01:02.660890 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc903d0-b009-49b7-9ab5-e195ea2b1224 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:01:02.670692 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc903d0-b009-49b7-9ab5-e195ea2b1224 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:01:02.672382 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc903d0-b009-49b7-9ab5-e195ea2b1224 service nova] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:01:02.683211 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 691/1363] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:01:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 15:01:05.303727 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17066558-29bf-4f40-bddd-2972bfb4b217 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:01:05.304633 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17066558-29bf-4f40-bddd-2972bfb4b217 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:01:05.304980 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17066558-29bf-4f40-bddd-2972bfb4b217 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:01:05.306600 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 673/1364] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 15:01:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 15:01:05.312993 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298cacba-2a3c-4bab-8ce1-892b62b476df None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:01:05.313508 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298cacba-2a3c-4bab-8ce1-892b62b476df None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:01:05.313602 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-298cacba-2a3c-4bab-8ce1-892b62b476df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:01:05.334538 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-298cacba-2a3c-4bab-8ce1-892b62b476df None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:01:05.335845 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-298cacba-2a3c-4bab-8ce1-892b62b476df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:01:05.344008 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 692/1365] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 15:01:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:01:05.393385 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4faefd0b-a487-4234-ba0e-08214afaccc4 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:01:05.393884 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4faefd0b-a487-4234-ba0e-08214afaccc4 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:01:05.394283 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4faefd0b-a487-4234-ba0e-08214afaccc4 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:01:05.395735 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 674/1366] 10.222.0.46 () {54 vars in 961 bytes} [Thu May 26 15:01:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 15:01:05.401368 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b274cf-c866-4d23-826e-795b48ecb115 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:01:05.401683 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b274cf-c866-4d23-826e-795b48ecb115 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:01:05.401966 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b274cf-c866-4d23-826e-795b48ecb115 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:01:05.424017 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-58b274cf-c866-4d23-826e-795b48ecb115 None None] MFA Rules not processed for user `d6639353c70449d1b3d10cd6487c3b00`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:01:05.424333 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-58b274cf-c866-4d23-826e-795b48ecb115 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:01:05.452921 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 693/1367] 10.222.0.46 () {58 vars in 1058 bytes} [Thu May 26 15:01:05 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:01:05.461085 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4faefd0b-a487-4234-ba0e-08214afaccc4 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:01:05.477330 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4faefd0b-a487-4234-ba0e-08214afaccc4 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:01:05.483796 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a03e14e-9b56-427e-8050-33fbf26f4756 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6639353c70449d1b3d10cd6487c3b00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:01:05.484451 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a03e14e-9b56-427e-8050-33fbf26f4756 service nova] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:01:05.484738 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a03e14e-9b56-427e-8050-33fbf26f4756 service nova] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:01:05.485018 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a03e14e-9b56-427e-8050-33fbf26f4756 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:01:05.494347 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a03e14e-9b56-427e-8050-33fbf26f4756 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:01:05.495954 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a03e14e-9b56-427e-8050-33fbf26f4756 service nova] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:01:05.506845 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 675/1368] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:01:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 15:01:07.838987 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c926edc-6f5a-45d6-95b3-67608f7e40a3 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:01:07.839582 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c926edc-6f5a-45d6-95b3-67608f7e40a3 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:01:07.839582 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c926edc-6f5a-45d6-95b3-67608f7e40a3 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:01:07.841502 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 694/1369] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 15:01:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 15:01:07.847512 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3344c3dd-53bf-4564-a4c4-555a7375a124 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:01:07.848005 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3344c3dd-53bf-4564-a4c4-555a7375a124 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:01:07.848114 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3344c3dd-53bf-4564-a4c4-555a7375a124 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:01:07.867127 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-3344c3dd-53bf-4564-a4c4-555a7375a124 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:01:07.868329 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-3344c3dd-53bf-4564-a4c4-555a7375a124 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:01:07.876955 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 676/1370] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 15:01:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:01:07.921795 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c926edc-6f5a-45d6-95b3-67608f7e40a3 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:01:07.938360 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c926edc-6f5a-45d6-95b3-67608f7e40a3 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:01:07.944092 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd92621-4d9c-41fa-a93b-b6342d9ef19a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6639353c70449d1b3d10cd6487c3b00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:01:07.944841 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd92621-4d9c-41fa-a93b-b6342d9ef19a service nova] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:01:07.945199 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd92621-4d9c-41fa-a93b-b6342d9ef19a service nova] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:01:07.945575 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd92621-4d9c-41fa-a93b-b6342d9ef19a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:01:07.955884 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd92621-4d9c-41fa-a93b-b6342d9ef19a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:01:07.957525 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd92621-4d9c-41fa-a93b-b6342d9ef19a service nova] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:01:07.968846 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 695/1371] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:01:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 15:01:08.233322 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f24108-b0cc-40bf-8e0c-eb8b69b29e39 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:01:08.233755 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f24108-b0cc-40bf-8e0c-eb8b69b29e39 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:01:08.233923 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f24108-b0cc-40bf-8e0c-eb8b69b29e39 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:01:08.235751 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 677/1372] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 15:01:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 15:01:08.243551 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4ae47b-9b5c-4b53-89c8-9976ae8db8d7 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:01:08.244029 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4ae47b-9b5c-4b53-89c8-9976ae8db8d7 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:01:08.244556 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4ae47b-9b5c-4b53-89c8-9976ae8db8d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:01:08.266897 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-0f4ae47b-9b5c-4b53-89c8-9976ae8db8d7 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:01:08.268135 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-0f4ae47b-9b5c-4b53-89c8-9976ae8db8d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:01:08.279273 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 696/1373] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 15:01:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:01:08.599669 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a489932-00eb-439c-8da4-2859885c1b12 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:01:08.599669 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a489932-00eb-439c-8da4-2859885c1b12 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:01:08.599669 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a489932-00eb-439c-8da4-2859885c1b12 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:01:08.599669 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 678/1374] 10.222.0.46 () {54 vars in 963 bytes} [Thu May 26 15:01:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 15:01:08.612992 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25090503-1f1b-410b-a5e7-3b950401186b None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:01:08.612992 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25090503-1f1b-410b-a5e7-3b950401186b None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:01:08.612992 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25090503-1f1b-410b-a5e7-3b950401186b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:01:08.613537 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:01:08.666949 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-25090503-1f1b-410b-a5e7-3b950401186b None None] MFA Rules not processed for user `d5b5b35614a547d783e6e77ddb8c1c03`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:01:08.668289 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-25090503-1f1b-410b-a5e7-3b950401186b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:01:08.741216 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 697/1375] 10.222.0.46 () {58 vars in 1060 bytes} [Thu May 26 15:01:08 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:01:08.750674 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a489932-00eb-439c-8da4-2859885c1b12 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:01:08.762866 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a489932-00eb-439c-8da4-2859885c1b12 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:01:08.766457 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ab6fef-5fac-46c4-9ec4-01b435bd0f45 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5b5b35614a547d783e6e77ddb8c1c03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:01:08.767104 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ab6fef-5fac-46c4-9ec4-01b435bd0f45 service cinder] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:01:08.767432 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ab6fef-5fac-46c4-9ec4-01b435bd0f45 service cinder] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:01:08.767668 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ab6fef-5fac-46c4-9ec4-01b435bd0f45 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:01:08.780036 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ab6fef-5fac-46c4-9ec4-01b435bd0f45 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:01:08.781953 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ab6fef-5fac-46c4-9ec4-01b435bd0f45 service cinder] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:01:08.793123 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 679/1376] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:01:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 15:01:10.315538 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780354af-032e-4f64-8c7d-12e0ec1ab369 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:01:10.316100 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780354af-032e-4f64-8c7d-12e0ec1ab369 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:01:10.316100 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780354af-032e-4f64-8c7d-12e0ec1ab369 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:01:10.317566 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 698/1377] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 15:01:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 15:01:10.323843 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d9e5e8-d370-4370-8ceb-48e19f203009 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:01:10.324421 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d9e5e8-d370-4370-8ceb-48e19f203009 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:01:10.324552 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d9e5e8-d370-4370-8ceb-48e19f203009 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:01:10.344747 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-76d9e5e8-d370-4370-8ceb-48e19f203009 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:01:10.346021 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-76d9e5e8-d370-4370-8ceb-48e19f203009 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:01:10.354350 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 680/1378] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 15:01:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 15:01:10.412552 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780354af-032e-4f64-8c7d-12e0ec1ab369 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:01:10.428678 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780354af-032e-4f64-8c7d-12e0ec1ab369 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:01:10.434387 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0fad53-eefa-4df4-bbb6-8deba14691b4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6639353c70449d1b3d10cd6487c3b00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:01:10.435061 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0fad53-eefa-4df4-bbb6-8deba14691b4 service nova] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:01:10.435344 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0fad53-eefa-4df4-bbb6-8deba14691b4 service nova] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:01:10.435624 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0fad53-eefa-4df4-bbb6-8deba14691b4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:01:10.445249 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0fad53-eefa-4df4-bbb6-8deba14691b4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:01:10.446974 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0fad53-eefa-4df4-bbb6-8deba14691b4 service nova] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:01:10.457988 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 699/1379] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:01:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 15:01:12.796604 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e30ba54-dfd3-4554-a20b-dab0cbca7825 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:01:12.797187 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e30ba54-dfd3-4554-a20b-dab0cbca7825 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:01:12.797187 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e30ba54-dfd3-4554-a20b-dab0cbca7825 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:01:12.798579 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 681/1380] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 15:01:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 15:01:12.804411 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9a1666-6a13-4bfa-8cbe-6c4c91c60b6e None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:01:12.804931 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9a1666-6a13-4bfa-8cbe-6c4c91c60b6e None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:01:12.805015 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d9a1666-6a13-4bfa-8cbe-6c4c91c60b6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:01:12.824097 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-0d9a1666-6a13-4bfa-8cbe-6c4c91c60b6e None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:01:12.826353 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-0d9a1666-6a13-4bfa-8cbe-6c4c91c60b6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:01:12.834596 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 700/1381] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 15:01:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:01:12.881010 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e30ba54-dfd3-4554-a20b-dab0cbca7825 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:01:12.896492 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e30ba54-dfd3-4554-a20b-dab0cbca7825 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:01:12.901951 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01990f92-a0c7-4b5e-8a40-cbbcbdb7e2c3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6639353c70449d1b3d10cd6487c3b00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:01:12.902603 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01990f92-a0c7-4b5e-8a40-cbbcbdb7e2c3 service nova] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:01:12.902921 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01990f92-a0c7-4b5e-8a40-cbbcbdb7e2c3 service nova] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:01:12.903197 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01990f92-a0c7-4b5e-8a40-cbbcbdb7e2c3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:01:12.912012 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01990f92-a0c7-4b5e-8a40-cbbcbdb7e2c3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:01:12.913642 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01990f92-a0c7-4b5e-8a40-cbbcbdb7e2c3 service nova] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:01:12.929659 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 682/1382] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:01:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 15:01:15.224983 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c83606d-58ee-4b0a-9349-d6b47d5b9d40 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:01:15.225860 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c83606d-58ee-4b0a-9349-d6b47d5b9d40 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:01:15.226131 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c83606d-58ee-4b0a-9349-d6b47d5b9d40 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:01:15.227653 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 701/1383] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 15:01:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 15:01:15.233832 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57936232-59b2-4d91-92d1-7a1924c845f1 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:01:15.234651 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57936232-59b2-4d91-92d1-7a1924c845f1 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:01:15.234926 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57936232-59b2-4d91-92d1-7a1924c845f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:01:15.254278 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-57936232-59b2-4d91-92d1-7a1924c845f1 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:01:15.255476 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-57936232-59b2-4d91-92d1-7a1924c845f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:01:15.266531 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 683/1384] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 15:01:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:01:15.315933 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c83606d-58ee-4b0a-9349-d6b47d5b9d40 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:01:15.334406 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c83606d-58ee-4b0a-9349-d6b47d5b9d40 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:01:15.340259 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea409be0-31ab-4e30-8e5e-0e72fecc4b4d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6639353c70449d1b3d10cd6487c3b00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:01:15.341023 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea409be0-31ab-4e30-8e5e-0e72fecc4b4d service nova] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:01:15.341312 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea409be0-31ab-4e30-8e5e-0e72fecc4b4d service nova] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:01:15.341833 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea409be0-31ab-4e30-8e5e-0e72fecc4b4d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:01:15.351630 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea409be0-31ab-4e30-8e5e-0e72fecc4b4d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:01:15.353367 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea409be0-31ab-4e30-8e5e-0e72fecc4b4d service nova] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:01:15.364844 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 702/1385] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:01:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 15:01:17.741155 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87669be5-6141-448f-9737-f83b1ea14385 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:01:17.741826 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87669be5-6141-448f-9737-f83b1ea14385 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:01:17.742393 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87669be5-6141-448f-9737-f83b1ea14385 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:01:17.744733 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 684/1386] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 15:01:17 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 15:01:17.750862 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8872e6b-38a8-4bb1-8946-c628fe7823ee None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:01:17.751890 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8872e6b-38a8-4bb1-8946-c628fe7823ee None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:01:17.751890 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8872e6b-38a8-4bb1-8946-c628fe7823ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:01:17.779916 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-e8872e6b-38a8-4bb1-8946-c628fe7823ee None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:01:17.782093 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-e8872e6b-38a8-4bb1-8946-c628fe7823ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:01:17.793183 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 703/1387] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 15:01:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 15:01:17.856235 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87669be5-6141-448f-9737-f83b1ea14385 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:01:17.875267 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87669be5-6141-448f-9737-f83b1ea14385 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:01:17.882142 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62996e35-9866-4cae-9f28-991b03244016 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6639353c70449d1b3d10cd6487c3b00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:01:17.882813 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62996e35-9866-4cae-9f28-991b03244016 service nova] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:01:17.883348 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62996e35-9866-4cae-9f28-991b03244016 service nova] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:01:17.883511 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62996e35-9866-4cae-9f28-991b03244016 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:01:17.895601 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62996e35-9866-4cae-9f28-991b03244016 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:01:17.897232 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62996e35-9866-4cae-9f28-991b03244016 service nova] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:01:17.907924 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 685/1388] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:01:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 15:01:20.353675 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e41b8f-0cd6-413b-bbca-a94151f957c7 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:01:20.354253 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e41b8f-0cd6-413b-bbca-a94151f957c7 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:01:20.354253 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e41b8f-0cd6-413b-bbca-a94151f957c7 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:01:20.355717 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 704/1389] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 15:01:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 15:01:20.361788 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c028c0e-d390-4356-8fed-008e8cd17592 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:01:20.362259 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c028c0e-d390-4356-8fed-008e8cd17592 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:01:20.362584 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c028c0e-d390-4356-8fed-008e8cd17592 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:01:20.391738 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-5c028c0e-d390-4356-8fed-008e8cd17592 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:01:20.393350 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-5c028c0e-d390-4356-8fed-008e8cd17592 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:01:20.404138 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 686/1390] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 15:01:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:01:20.456854 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e41b8f-0cd6-413b-bbca-a94151f957c7 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:01:20.473433 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e41b8f-0cd6-413b-bbca-a94151f957c7 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:01:20.480867 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b1e2e1-40bf-4501-b4b8-55b96cd87458 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6639353c70449d1b3d10cd6487c3b00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:01:20.481679 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b1e2e1-40bf-4501-b4b8-55b96cd87458 service nova] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:01:20.482065 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b1e2e1-40bf-4501-b4b8-55b96cd87458 service nova] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:01:20.482384 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b1e2e1-40bf-4501-b4b8-55b96cd87458 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:01:20.493152 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b1e2e1-40bf-4501-b4b8-55b96cd87458 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:01:20.495135 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b1e2e1-40bf-4501-b4b8-55b96cd87458 service nova] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:01:20.516989 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 705/1391] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:01:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 15:01:22.942132 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc620d6-3219-41e1-9671-42dca059a667 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:01:22.942656 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc620d6-3219-41e1-9671-42dca059a667 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:01:22.942916 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc620d6-3219-41e1-9671-42dca059a667 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:01:22.944624 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 687/1392] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 15:01:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 15:01:22.951694 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915969be-8b43-402b-a147-a6da3ad507c3 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:01:22.952167 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915969be-8b43-402b-a147-a6da3ad507c3 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:01:22.952514 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915969be-8b43-402b-a147-a6da3ad507c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:01:22.978596 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-915969be-8b43-402b-a147-a6da3ad507c3 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:01:22.979862 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-915969be-8b43-402b-a147-a6da3ad507c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:01:22.988835 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 706/1393] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 15:01:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:01:23.036116 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc620d6-3219-41e1-9671-42dca059a667 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:01:23.053972 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc620d6-3219-41e1-9671-42dca059a667 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:01:23.061557 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be810ab3-b4f5-4e86-ae29-8c5d992872d5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6639353c70449d1b3d10cd6487c3b00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:01:23.062363 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be810ab3-b4f5-4e86-ae29-8c5d992872d5 service nova] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:01:23.062726 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be810ab3-b4f5-4e86-ae29-8c5d992872d5 service nova] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:01:23.063080 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be810ab3-b4f5-4e86-ae29-8c5d992872d5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:01:23.074659 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be810ab3-b4f5-4e86-ae29-8c5d992872d5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:01:23.076710 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be810ab3-b4f5-4e86-ae29-8c5d992872d5 service nova] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:01:23.093682 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 688/1394] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:01:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 15:01:25.450937 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3637c080-54e9-4c15-9a32-ef27909718d9 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:01:25.451542 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3637c080-54e9-4c15-9a32-ef27909718d9 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:01:25.451542 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3637c080-54e9-4c15-9a32-ef27909718d9 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:01:25.452896 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 707/1395] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 15:01:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 15:01:25.458934 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fa5ef4-0880-449a-8b2e-61dbea13b636 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:01:25.459497 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fa5ef4-0880-449a-8b2e-61dbea13b636 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:01:25.459497 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fa5ef4-0880-449a-8b2e-61dbea13b636 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:01:25.466299 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=52 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:01:25.481637 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-27fa5ef4-0880-449a-8b2e-61dbea13b636 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:01:25.482868 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-27fa5ef4-0880-449a-8b2e-61dbea13b636 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:01:25.491829 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 689/1396] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 15:01:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:01:25.537745 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3637c080-54e9-4c15-9a32-ef27909718d9 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:01:25.555990 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3637c080-54e9-4c15-9a32-ef27909718d9 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:01:25.561557 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec23a0b4-9297-4977-b5d3-c463b61b45e9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6639353c70449d1b3d10cd6487c3b00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:01:25.562218 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec23a0b4-9297-4977-b5d3-c463b61b45e9 service nova] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:01:25.562505 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec23a0b4-9297-4977-b5d3-c463b61b45e9 service nova] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:01:25.562781 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec23a0b4-9297-4977-b5d3-c463b61b45e9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:01:25.572306 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec23a0b4-9297-4977-b5d3-c463b61b45e9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:01:25.574364 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec23a0b4-9297-4977-b5d3-c463b61b45e9 service nova] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:01:25.588478 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 708/1397] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:01:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 15:01:25.718718 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fb6c93-c277-4e73-8271-d5c11f773b58 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:01:25.719003 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fb6c93-c277-4e73-8271-d5c11f773b58 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:01:25.719291 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fb6c93-c277-4e73-8271-d5c11f773b58 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:01:25.720689 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 690/1398] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 15:01:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 15:01:25.726915 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d93861-f39d-4a9b-a7cb-0f9e86e19d55 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:01:25.727301 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d93861-f39d-4a9b-a7cb-0f9e86e19d55 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:01:25.727513 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d93861-f39d-4a9b-a7cb-0f9e86e19d55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:01:25.747325 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-d3d93861-f39d-4a9b-a7cb-0f9e86e19d55 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:01:25.748564 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-d3d93861-f39d-4a9b-a7cb-0f9e86e19d55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:01:25.757099 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 709/1399] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 15:01:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:01:25.817122 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990230de-af2c-407e-8887-10f384ef7016 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:01:25.817536 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990230de-af2c-407e-8887-10f384ef7016 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:01:25.817891 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990230de-af2c-407e-8887-10f384ef7016 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:01:25.819597 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 691/1400] 10.222.0.46 () {54 vars in 962 bytes} [Thu May 26 15:01:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 15:01:25.825826 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc788e5e-fd51-46e4-ab91-deaa0500f10b None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:01:25.826294 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc788e5e-fd51-46e4-ab91-deaa0500f10b None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:01:25.826579 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc788e5e-fd51-46e4-ab91-deaa0500f10b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:01:25.887516 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-bc788e5e-fd51-46e4-ab91-deaa0500f10b None None] MFA Rules not processed for user `0a436dbdee624f44ad76cca8416b0ad1`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:01:25.888764 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-bc788e5e-fd51-46e4-ab91-deaa0500f10b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:01:25.924550 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:01:25.956561 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 710/1401] 10.222.0.46 () {58 vars in 1059 bytes} [Thu May 26 15:01:25 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:01:25.964477 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990230de-af2c-407e-8887-10f384ef7016 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:01:25.976037 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990230de-af2c-407e-8887-10f384ef7016 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:01:25.979715 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754c302b-9988-4d19-9197-42ce5494dce2 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a436dbdee624f44ad76cca8416b0ad1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:01:25.980395 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754c302b-9988-4d19-9197-42ce5494dce2 service glance] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:01:25.980673 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754c302b-9988-4d19-9197-42ce5494dce2 service glance] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:01:25.980944 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754c302b-9988-4d19-9197-42ce5494dce2 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:01:25.990657 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754c302b-9988-4d19-9197-42ce5494dce2 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:01:25.992110 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754c302b-9988-4d19-9197-42ce5494dce2 service glance] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:01:26.002649 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 692/1402] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:01:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 15:01:26.564676 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5aa1da6-81d0-4a0d-a44d-b1ba3fa44f0d None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:01:26.565236 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5aa1da6-81d0-4a0d-a44d-b1ba3fa44f0d None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:01:26.565236 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5aa1da6-81d0-4a0d-a44d-b1ba3fa44f0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:01:26.616702 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-c5aa1da6-81d0-4a0d-a44d-b1ba3fa44f0d None None] MFA Rules not processed for user `8028c520d93449a6a087c281280ee13d`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:01:26.617982 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-c5aa1da6-81d0-4a0d-a44d-b1ba3fa44f0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:01:26.687979 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 711/1403] 10.222.0.46 () {58 vars in 1009 bytes} [Thu May 26 15:01:26 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:01:26.735744 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f28e37-84e9-4ec5-9548-1c1fa3720de8 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:01:26.736268 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f28e37-84e9-4ec5-9548-1c1fa3720de8 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:01:26.736365 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f28e37-84e9-4ec5-9548-1c1fa3720de8 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:01:26.737699 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 693/1404] 10.222.0.46 () {54 vars in 942 bytes} [Thu May 26 15:01:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 15:01:26.744323 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16039eae-8021-46a2-96b5-67cb27ffc0a4 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:01:26.744691 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16039eae-8021-46a2-96b5-67cb27ffc0a4 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:01:26.745041 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16039eae-8021-46a2-96b5-67cb27ffc0a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:01:26.800694 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-16039eae-8021-46a2-96b5-67cb27ffc0a4 None None] MFA Rules not processed for user `a4bea272624b45d8ac2ad27bd07df885`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:01:26.801988 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-16039eae-8021-46a2-96b5-67cb27ffc0a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:01:26.881734 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 712/1405] 10.222.0.46 () {58 vars in 1039 bytes} [Thu May 26 15:01:26 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:01:26.891388 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f28e37-84e9-4ec5-9548-1c1fa3720de8 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:01:26.905547 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f28e37-84e9-4ec5-9548-1c1fa3720de8 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:01:26.910399 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ae9994-b71f-4478-ba88-c6187db11e52 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4bea272624b45d8ac2ad27bd07df885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:01:26.911028 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ae9994-b71f-4478-ba88-c6187db11e52 service swift] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:01:26.911311 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ae9994-b71f-4478-ba88-c6187db11e52 service swift] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:01:26.911585 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ae9994-b71f-4478-ba88-c6187db11e52 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:01:26.920192 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ae9994-b71f-4478-ba88-c6187db11e52 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:01:26.921842 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ae9994-b71f-4478-ba88-c6187db11e52 service swift] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:01:26.929930 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 694/1406] 10.222.0.46 () {60 vars in 1384 bytes} [Thu May 26 15:01:26 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 26 15:01:27.890655 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7501d4-34a0-4427-a7cf-fb70aa531eec None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:01:27.895052 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7501d4-34a0-4427-a7cf-fb70aa531eec None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:01:27.895052 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7501d4-34a0-4427-a7cf-fb70aa531eec None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:01:27.895052 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 713/1407] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 15:01:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 15:01:27.901590 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99c52f8-79b7-4d31-93ee-12f9d0b223c8 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:01:27.901590 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99c52f8-79b7-4d31-93ee-12f9d0b223c8 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:01:27.901590 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99c52f8-79b7-4d31-93ee-12f9d0b223c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:01:27.923372 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-f99c52f8-79b7-4d31-93ee-12f9d0b223c8 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:01:27.924613 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-f99c52f8-79b7-4d31-93ee-12f9d0b223c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:01:27.937534 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 695/1408] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 15:01:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:01:27.983399 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7501d4-34a0-4427-a7cf-fb70aa531eec None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:01:28.006398 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7501d4-34a0-4427-a7cf-fb70aa531eec None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:01:28.015132 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb011082-960c-453d-948d-1685a65ea00d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6639353c70449d1b3d10cd6487c3b00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:01:28.015811 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb011082-960c-453d-948d-1685a65ea00d service nova] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:01:28.016091 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb011082-960c-453d-948d-1685a65ea00d service nova] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:01:28.016365 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb011082-960c-453d-948d-1685a65ea00d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:01:28.027285 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb011082-960c-453d-948d-1685a65ea00d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:01:28.029511 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb011082-960c-453d-948d-1685a65ea00d service nova] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:01:28.036631 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:01:28.045231 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 714/1409] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:01:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 15:01:37.917192 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=17 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:01:50.272400 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba5eea5-9f4d-48b5-aaa6-df81661ac70f None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:01:50.273155 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba5eea5-9f4d-48b5-aaa6-df81661ac70f None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:01:50.273155 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba5eea5-9f4d-48b5-aaa6-df81661ac70f None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:01:50.274347 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 696/1410] 10.222.0.46 () {54 vars in 918 bytes} [Thu May 26 15:01:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 15:01:50.279558 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00bf142-e737-4f87-9e2e-3bb4cd9a2f42 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:01:50.280068 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00bf142-e737-4f87-9e2e-3bb4cd9a2f42 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:01:50.280172 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00bf142-e737-4f87-9e2e-3bb4cd9a2f42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:01:50.298063 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-c00bf142-e737-4f87-9e2e-3bb4cd9a2f42 None None] MFA Rules not processed for user `bf55a00e3b084aa3b22c25a90e6a773c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:01:50.299280 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-c00bf142-e737-4f87-9e2e-3bb4cd9a2f42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:01:50.308415 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 715/1411] 10.222.0.46 () {58 vars in 1015 bytes} [Thu May 26 15:01:50 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:01:50.323139 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c6894c-44ff-4cd2-9fb5-0241f883e8a4 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:01:50.324010 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c6894c-44ff-4cd2-9fb5-0241f883e8a4 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:01:50.324292 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c6894c-44ff-4cd2-9fb5-0241f883e8a4 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:01:50.325686 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 697/1412] 10.222.0.46 () {54 vars in 918 bytes} [Thu May 26 15:01:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 15:01:50.330366 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1b5e6a-3c65-4ee7-9122-fce01f8b7da0 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:01:50.330658 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1b5e6a-3c65-4ee7-9122-fce01f8b7da0 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:01:50.330936 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1b5e6a-3c65-4ee7-9122-fce01f8b7da0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:01:50.349117 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-bc1b5e6a-3c65-4ee7-9122-fce01f8b7da0 None None] MFA Rules not processed for user `d6639353c70449d1b3d10cd6487c3b00`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:01:50.350364 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-bc1b5e6a-3c65-4ee7-9122-fce01f8b7da0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:01:50.359802 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 716/1413] 10.222.0.46 () {58 vars in 1015 bytes} [Thu May 26 15:01:50 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:01:53.080448 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20955aef-eac4-40db-9913-c89e957c0348 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:01:53.081009 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20955aef-eac4-40db-9913-c89e957c0348 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:01:53.081009 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20955aef-eac4-40db-9913-c89e957c0348 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:01:53.082688 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 698/1414] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 26 15:01:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 15:01:53.087695 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d07ea85-9f8b-4278-9f6d-3182124c1bd2 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:01:53.088130 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d07ea85-9f8b-4278-9f6d-3182124c1bd2 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:01:53.088245 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d07ea85-9f8b-4278-9f6d-3182124c1bd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:01:53.107665 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-1d07ea85-9f8b-4278-9f6d-3182124c1bd2 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:01:53.109024 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-1d07ea85-9f8b-4278-9f6d-3182124c1bd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:01:53.117362 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 717/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 26 15:01:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:01:53.164833 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20955aef-eac4-40db-9913-c89e957c0348 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:01:53.179774 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20955aef-eac4-40db-9913-c89e957c0348 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:01:53.185169 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6ba903-40c3-4306-befc-6d511cae09df service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6639353c70449d1b3d10cd6487c3b00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:01:53.185833 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6ba903-40c3-4306-befc-6d511cae09df service nova] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:01:53.186120 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6ba903-40c3-4306-befc-6d511cae09df service nova] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:01:53.186399 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6ba903-40c3-4306-befc-6d511cae09df service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:01:53.194998 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6ba903-40c3-4306-befc-6d511cae09df service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:01:53.196593 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6ba903-40c3-4306-befc-6d511cae09df service nova] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:01:53.206161 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 699/1416] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:01:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 15:01:54.174446 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a37597-8034-4679-988d-3d9a293bb2af None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:01:54.175028 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a37597-8034-4679-988d-3d9a293bb2af None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:01:54.175028 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a37597-8034-4679-988d-3d9a293bb2af None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:01:54.176358 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 718/1417] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 26 15:01:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 15:01:54.181375 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4405b77b-9560-4ae3-8b5c-fe4ba2c40910 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:01:54.182008 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4405b77b-9560-4ae3-8b5c-fe4ba2c40910 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:01:54.182008 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4405b77b-9560-4ae3-8b5c-fe4ba2c40910 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:01:54.200323 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-4405b77b-9560-4ae3-8b5c-fe4ba2c40910 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:01:54.201566 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-4405b77b-9560-4ae3-8b5c-fe4ba2c40910 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:01:54.209747 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 700/1418] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 26 15:01:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:01:54.376319 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a37597-8034-4679-988d-3d9a293bb2af None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:01:54.392002 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a37597-8034-4679-988d-3d9a293bb2af None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:01:54.397380 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbebbfbe-1405-43fe-bee0-fce6f19d5e50 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6639353c70449d1b3d10cd6487c3b00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:01:54.398048 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbebbfbe-1405-43fe-bee0-fce6f19d5e50 service nova] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:01:54.398340 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbebbfbe-1405-43fe-bee0-fce6f19d5e50 service nova] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:01:54.398619 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbebbfbe-1405-43fe-bee0-fce6f19d5e50 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:01:54.407465 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbebbfbe-1405-43fe-bee0-fce6f19d5e50 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:01:54.409046 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbebbfbe-1405-43fe-bee0-fce6f19d5e50 service nova] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:01:54.419525 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 719/1419] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:01:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 15:01:55.376262 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4784c75d-336e-4a81-912a-9b581617bb7d None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:01:55.377116 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4784c75d-336e-4a81-912a-9b581617bb7d None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:01:55.377116 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4784c75d-336e-4a81-912a-9b581617bb7d None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:01:55.378507 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 701/1420] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 26 15:01:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 15:01:55.383878 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded4fe87-a0c3-41a1-9f32-4739b550f0f5 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:01:55.384320 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded4fe87-a0c3-41a1-9f32-4739b550f0f5 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:01:55.384456 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded4fe87-a0c3-41a1-9f32-4739b550f0f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:01:55.402906 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-ded4fe87-a0c3-41a1-9f32-4739b550f0f5 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:01:55.404112 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-ded4fe87-a0c3-41a1-9f32-4739b550f0f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:01:55.411884 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 720/1421] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 26 15:01:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 15:01:55.564829 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4784c75d-336e-4a81-912a-9b581617bb7d None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:01:55.579426 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4784c75d-336e-4a81-912a-9b581617bb7d None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:01:55.584643 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ce4608-ed74-4b7b-9125-d8387c4b423b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6639353c70449d1b3d10cd6487c3b00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:01:55.585288 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ce4608-ed74-4b7b-9125-d8387c4b423b service nova] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:01:55.585611 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ce4608-ed74-4b7b-9125-d8387c4b423b service nova] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:01:55.585887 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ce4608-ed74-4b7b-9125-d8387c4b423b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:01:55.595134 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ce4608-ed74-4b7b-9125-d8387c4b423b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:01:55.596737 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ce4608-ed74-4b7b-9125-d8387c4b423b service nova] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:01:55.607264 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 702/1422] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:01:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 15:01:56.527131 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e6684c-b8a4-4c9b-b301-3d9122e31570 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:01:56.527718 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e6684c-b8a4-4c9b-b301-3d9122e31570 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:01:56.527718 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e6684c-b8a4-4c9b-b301-3d9122e31570 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:01:56.529134 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 721/1423] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 26 15:01:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 15:01:56.534578 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d598a9fa-62e4-45b7-bba4-8a363320fb25 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:01:56.535356 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d598a9fa-62e4-45b7-bba4-8a363320fb25 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:01:56.535356 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d598a9fa-62e4-45b7-bba4-8a363320fb25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:01:56.561562 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-d598a9fa-62e4-45b7-bba4-8a363320fb25 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:01:56.562772 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-d598a9fa-62e4-45b7-bba4-8a363320fb25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:01:56.570735 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 703/1424] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 26 15:01:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:01:56.721816 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e6684c-b8a4-4c9b-b301-3d9122e31570 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:01:56.736977 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e6684c-b8a4-4c9b-b301-3d9122e31570 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:01:56.742464 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fcfac9-0041-4d12-afa2-24b6269f12f4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6639353c70449d1b3d10cd6487c3b00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:01:56.743136 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fcfac9-0041-4d12-afa2-24b6269f12f4 service nova] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:01:56.743418 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fcfac9-0041-4d12-afa2-24b6269f12f4 service nova] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:01:56.743691 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fcfac9-0041-4d12-afa2-24b6269f12f4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:01:56.753297 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fcfac9-0041-4d12-afa2-24b6269f12f4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:01:56.754927 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fcfac9-0041-4d12-afa2-24b6269f12f4 service nova] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:01:56.764714 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 722/1425] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:01:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 15:01:57.689964 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc8f3bd-4239-4716-abb2-d59ce3738d75 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:01:57.690518 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc8f3bd-4239-4716-abb2-d59ce3738d75 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:01:57.690518 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc8f3bd-4239-4716-abb2-d59ce3738d75 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:01:57.691949 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 704/1426] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 26 15:01:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 15:01:57.697609 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b9473d-2006-4fa3-b672-069d9f72f328 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:01:57.698154 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b9473d-2006-4fa3-b672-069d9f72f328 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:01:57.698828 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b9473d-2006-4fa3-b672-069d9f72f328 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:01:57.717134 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-a4b9473d-2006-4fa3-b672-069d9f72f328 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:01:57.718405 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-a4b9473d-2006-4fa3-b672-069d9f72f328 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:01:57.726417 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 723/1427] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 26 15:01:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:01:57.876224 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc8f3bd-4239-4716-abb2-d59ce3738d75 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:01:57.891390 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc8f3bd-4239-4716-abb2-d59ce3738d75 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:01:57.896740 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8108881e-810b-41f3-8164-9cb0e8545095 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6639353c70449d1b3d10cd6487c3b00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:01:57.897390 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8108881e-810b-41f3-8164-9cb0e8545095 service nova] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:01:57.897707 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8108881e-810b-41f3-8164-9cb0e8545095 service nova] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:01:57.897998 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8108881e-810b-41f3-8164-9cb0e8545095 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:01:57.906694 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8108881e-810b-41f3-8164-9cb0e8545095 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:01:57.908293 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8108881e-810b-41f3-8164-9cb0e8545095 service nova] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:01:57.918157 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 705/1428] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:01:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 15:01:58.782717 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c79b261-50dc-4767-be5f-d77e67eb6d0b None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:01:58.783328 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c79b261-50dc-4767-be5f-d77e67eb6d0b None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:01:58.783328 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c79b261-50dc-4767-be5f-d77e67eb6d0b None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:01:58.784679 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 724/1429] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 26 15:01:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 15:01:58.789907 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f27259-51e8-4fe8-9f7b-181e3a32ec4a None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:01:58.790302 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f27259-51e8-4fe8-9f7b-181e3a32ec4a None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:01:58.790464 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f27259-51e8-4fe8-9f7b-181e3a32ec4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:01:58.809466 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-84f27259-51e8-4fe8-9f7b-181e3a32ec4a None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:01:58.810676 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-84f27259-51e8-4fe8-9f7b-181e3a32ec4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:01:58.818562 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 706/1430] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 26 15:01:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:01:58.968201 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c79b261-50dc-4767-be5f-d77e67eb6d0b None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:01:58.983090 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c79b261-50dc-4767-be5f-d77e67eb6d0b None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:01:58.988714 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405bc094-3c39-46df-9d59-83530178b8b3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6639353c70449d1b3d10cd6487c3b00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:01:58.989368 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405bc094-3c39-46df-9d59-83530178b8b3 service nova] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:01:58.989683 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405bc094-3c39-46df-9d59-83530178b8b3 service nova] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:01:58.989964 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405bc094-3c39-46df-9d59-83530178b8b3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:01:58.999357 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405bc094-3c39-46df-9d59-83530178b8b3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:01:59.000963 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405bc094-3c39-46df-9d59-83530178b8b3 service nova] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:01:59.011042 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 725/1431] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:01:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 15:01:59.889019 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47d5642-ac75-4cbd-8c22-50941ab428b6 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:01:59.889592 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47d5642-ac75-4cbd-8c22-50941ab428b6 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:01:59.889592 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47d5642-ac75-4cbd-8c22-50941ab428b6 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:01:59.891073 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 707/1432] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 26 15:01:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 15:01:59.898409 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78adcedf-4299-4b2b-a951-1349ad9054d3 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:01:59.898862 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78adcedf-4299-4b2b-a951-1349ad9054d3 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:01:59.898972 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78adcedf-4299-4b2b-a951-1349ad9054d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:01:59.917987 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-78adcedf-4299-4b2b-a951-1349ad9054d3 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:01:59.919216 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-78adcedf-4299-4b2b-a951-1349ad9054d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:01:59.927226 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 726/1433] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 26 15:01:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:02:00.079487 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47d5642-ac75-4cbd-8c22-50941ab428b6 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:02:00.094072 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47d5642-ac75-4cbd-8c22-50941ab428b6 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:02:00.099640 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8560e3f5-48d6-4e49-940b-4f13fa8b76a2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6639353c70449d1b3d10cd6487c3b00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:02:00.100288 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8560e3f5-48d6-4e49-940b-4f13fa8b76a2 service nova] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:02:00.100568 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8560e3f5-48d6-4e49-940b-4f13fa8b76a2 service nova] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:02:00.100843 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8560e3f5-48d6-4e49-940b-4f13fa8b76a2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:02:00.110121 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8560e3f5-48d6-4e49-940b-4f13fa8b76a2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:02:00.111731 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8560e3f5-48d6-4e49-940b-4f13fa8b76a2 service nova] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:02:00.122298 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 708/1434] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:02:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 15:02:00.998568 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463dc830-8a56-440c-9864-9e6035347396 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:02:00.999155 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463dc830-8a56-440c-9864-9e6035347396 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:02:00.999526 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463dc830-8a56-440c-9864-9e6035347396 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:02:01.001371 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 727/1435] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 26 15:02:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 15:02:01.009018 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016ca96a-6fff-4f51-9254-38cff112f7d2 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:02:01.009546 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016ca96a-6fff-4f51-9254-38cff112f7d2 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:02:01.009653 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016ca96a-6fff-4f51-9254-38cff112f7d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:02:01.030149 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-016ca96a-6fff-4f51-9254-38cff112f7d2 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:02:01.031385 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-016ca96a-6fff-4f51-9254-38cff112f7d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:02:01.039343 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 709/1436] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 26 15:02:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:02:01.191460 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463dc830-8a56-440c-9864-9e6035347396 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:02:01.206330 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463dc830-8a56-440c-9864-9e6035347396 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:02:01.211891 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e26cdf-607c-47a6-9087-ca0967446d30 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6639353c70449d1b3d10cd6487c3b00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:02:01.212557 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e26cdf-607c-47a6-9087-ca0967446d30 service nova] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:02:01.212869 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e26cdf-607c-47a6-9087-ca0967446d30 service nova] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:02:01.213142 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e26cdf-607c-47a6-9087-ca0967446d30 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:02:01.222108 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e26cdf-607c-47a6-9087-ca0967446d30 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:02:01.223718 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e26cdf-607c-47a6-9087-ca0967446d30 service nova] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:02:01.234303 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 728/1437] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:02:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 15:02:02.106498 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d790d9-2372-4e44-9cbc-b78b6ca6acaf None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:02:02.107064 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d790d9-2372-4e44-9cbc-b78b6ca6acaf None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:02:02.107064 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d790d9-2372-4e44-9cbc-b78b6ca6acaf None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:02:02.108430 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 710/1438] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 26 15:02:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 15:02:02.113579 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90955cc2-b80a-4b34-9aa5-72d1568c78e4 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:02:02.114020 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90955cc2-b80a-4b34-9aa5-72d1568c78e4 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:02:02.114208 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90955cc2-b80a-4b34-9aa5-72d1568c78e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:02:02.132828 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-90955cc2-b80a-4b34-9aa5-72d1568c78e4 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:02:02.134056 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-90955cc2-b80a-4b34-9aa5-72d1568c78e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:02:02.142457 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 729/1439] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 26 15:02:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 15:02:02.294457 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d790d9-2372-4e44-9cbc-b78b6ca6acaf None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:02:02.309148 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d790d9-2372-4e44-9cbc-b78b6ca6acaf None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:02:02.314851 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66bf17b-d7e5-435f-a603-2af39556c57a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6639353c70449d1b3d10cd6487c3b00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:02:02.315497 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66bf17b-d7e5-435f-a603-2af39556c57a service nova] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:02:02.315793 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66bf17b-d7e5-435f-a603-2af39556c57a service nova] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:02:02.316067 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66bf17b-d7e5-435f-a603-2af39556c57a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:02:02.326254 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66bf17b-d7e5-435f-a603-2af39556c57a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:02:02.328200 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66bf17b-d7e5-435f-a603-2af39556c57a service nova] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:02:02.338333 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 711/1440] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:02:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 15:02:04.418480 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caba90f7-a5b7-4e68-a28c-22725ecbdd26 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:02:04.419025 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caba90f7-a5b7-4e68-a28c-22725ecbdd26 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:02:04.419025 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caba90f7-a5b7-4e68-a28c-22725ecbdd26 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:02:04.420403 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 730/1441] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 26 15:02:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 15:02:04.426027 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a6b3ab-82e8-43ca-8f2c-3b512c407da7 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:02:04.426497 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a6b3ab-82e8-43ca-8f2c-3b512c407da7 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:02:04.426611 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a6b3ab-82e8-43ca-8f2c-3b512c407da7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:02:04.445200 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-e0a6b3ab-82e8-43ca-8f2c-3b512c407da7 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:02:04.446439 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-e0a6b3ab-82e8-43ca-8f2c-3b512c407da7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:02:04.454468 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 712/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 26 15:02:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:02:04.494911 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caba90f7-a5b7-4e68-a28c-22725ecbdd26 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:02:04.496747 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-caba90f7-a5b7-4e68-a28c-22725ecbdd26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:02:04.512248 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caba90f7-a5b7-4e68-a28c-22725ecbdd26 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:02:04.517985 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed97165e-4af6-4934-b289-4f8ae4c7450f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0156a2f6f3794f479e76b85bdbbe5c26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:02:04.518640 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed97165e-4af6-4934-b289-4f8ae4c7450f service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:02:04.518922 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed97165e-4af6-4934-b289-4f8ae4c7450f service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:02:04.519196 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed97165e-4af6-4934-b289-4f8ae4c7450f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:02:04.531678 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed97165e-4af6-4934-b289-4f8ae4c7450f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:02:04.533118 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed97165e-4af6-4934-b289-4f8ae4c7450f service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:02:04.543736 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 731/1443] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:02:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 15:02:06.968145 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8094ce8c-8f3e-4394-9b73-f812d60d9094 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:02:06.968672 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8094ce8c-8f3e-4394-9b73-f812d60d9094 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:02:06.968672 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8094ce8c-8f3e-4394-9b73-f812d60d9094 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:02:06.970140 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 713/1444] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 26 15:02:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 15:02:06.975685 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73465ea-3754-47a8-98fc-c32cc3d76482 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:02:06.976229 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73465ea-3754-47a8-98fc-c32cc3d76482 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:02:06.976362 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73465ea-3754-47a8-98fc-c32cc3d76482 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:02:06.995811 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-b73465ea-3754-47a8-98fc-c32cc3d76482 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:02:06.997038 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-b73465ea-3754-47a8-98fc-c32cc3d76482 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:02:07.005240 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 732/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 26 15:02:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:02:07.044186 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8094ce8c-8f3e-4394-9b73-f812d60d9094 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:02:07.054991 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8094ce8c-8f3e-4394-9b73-f812d60d9094 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:02:07.059203 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e547abf9-4fa7-407e-8d54-77d6f61bc1db service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0156a2f6f3794f479e76b85bdbbe5c26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:02:07.059847 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e547abf9-4fa7-407e-8d54-77d6f61bc1db service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:02:07.060123 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e547abf9-4fa7-407e-8d54-77d6f61bc1db service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:02:07.060394 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e547abf9-4fa7-407e-8d54-77d6f61bc1db service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:02:07.070818 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e547abf9-4fa7-407e-8d54-77d6f61bc1db service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:02:07.072253 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e547abf9-4fa7-407e-8d54-77d6f61bc1db service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:02:07.082368 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 714/1446] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:02:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 15:02:09.654633 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba37c053-f004-4c5f-a55c-5f5b73e54104 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:02:09.655161 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba37c053-f004-4c5f-a55c-5f5b73e54104 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:02:09.655277 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba37c053-f004-4c5f-a55c-5f5b73e54104 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:02:09.656868 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 733/1447] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 26 15:02:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 15:02:09.662860 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d58d86-3174-4b64-8333-d058a4aebfc9 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:02:09.663363 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d58d86-3174-4b64-8333-d058a4aebfc9 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:02:09.663457 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d58d86-3174-4b64-8333-d058a4aebfc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:02:09.682095 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-91d58d86-3174-4b64-8333-d058a4aebfc9 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:02:09.683321 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-91d58d86-3174-4b64-8333-d058a4aebfc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:02:09.691552 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 715/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 26 15:02:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:02:09.730785 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba37c053-f004-4c5f-a55c-5f5b73e54104 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:02:09.741863 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba37c053-f004-4c5f-a55c-5f5b73e54104 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:02:09.745428 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbab8e0a-4f91-4981-b943-699e9da4f6ff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0156a2f6f3794f479e76b85bdbbe5c26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:02:09.746107 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbab8e0a-4f91-4981-b943-699e9da4f6ff service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:02:09.746394 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbab8e0a-4f91-4981-b943-699e9da4f6ff service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:02:09.746736 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbab8e0a-4f91-4981-b943-699e9da4f6ff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:02:09.764781 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbab8e0a-4f91-4981-b943-699e9da4f6ff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:02:09.766277 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbab8e0a-4f91-4981-b943-699e9da4f6ff service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:02:09.776868 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 734/1449] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:02:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 15:02:12.185764 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1dcc25-bbc4-4d3c-b6e1-a7fac4ad08c3 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:02:12.186373 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1dcc25-bbc4-4d3c-b6e1-a7fac4ad08c3 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:02:12.186373 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1dcc25-bbc4-4d3c-b6e1-a7fac4ad08c3 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:02:12.187678 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 716/1450] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 26 15:02:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 15:02:12.192907 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44168c83-75ee-4611-a27c-2b2d914a202c None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:02:12.193434 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44168c83-75ee-4611-a27c-2b2d914a202c None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:02:12.193568 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44168c83-75ee-4611-a27c-2b2d914a202c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:02:12.211692 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-44168c83-75ee-4611-a27c-2b2d914a202c None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:02:12.212902 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-44168c83-75ee-4611-a27c-2b2d914a202c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:02:12.221307 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 735/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 26 15:02:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:02:12.258869 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1dcc25-bbc4-4d3c-b6e1-a7fac4ad08c3 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:02:12.270129 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1dcc25-bbc4-4d3c-b6e1-a7fac4ad08c3 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:02:12.273682 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a5ff84-0279-478a-8b50-dfb077a40e91 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0156a2f6f3794f479e76b85bdbbe5c26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:02:12.274318 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a5ff84-0279-478a-8b50-dfb077a40e91 service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:02:12.274600 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a5ff84-0279-478a-8b50-dfb077a40e91 service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:02:12.274903 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a5ff84-0279-478a-8b50-dfb077a40e91 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:02:12.284258 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a5ff84-0279-478a-8b50-dfb077a40e91 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:02:12.285714 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a5ff84-0279-478a-8b50-dfb077a40e91 service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:02:12.294901 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 717/1452] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:02:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 15:02:14.524160 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b0ff23-0852-40c1-b3f4-134a8328427e None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:02:14.524738 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b0ff23-0852-40c1-b3f4-134a8328427e None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:02:14.524738 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b0ff23-0852-40c1-b3f4-134a8328427e None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:02:14.526129 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 736/1453] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 26 15:02:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 15:02:14.531311 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d859f084-0aca-4d2b-8fb7-fab207e52739 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:02:14.531741 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d859f084-0aca-4d2b-8fb7-fab207e52739 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:02:14.531968 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d859f084-0aca-4d2b-8fb7-fab207e52739 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:02:14.550503 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-d859f084-0aca-4d2b-8fb7-fab207e52739 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:02:14.551749 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-d859f084-0aca-4d2b-8fb7-fab207e52739 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:02:14.560257 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 718/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 26 15:02:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:02:14.597154 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b0ff23-0852-40c1-b3f4-134a8328427e None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:02:14.610881 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b0ff23-0852-40c1-b3f4-134a8328427e None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:02:14.616666 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d24a53-be32-4edd-805b-99c559d3cba3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0156a2f6f3794f479e76b85bdbbe5c26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:02:14.617677 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d24a53-be32-4edd-805b-99c559d3cba3 service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:02:14.617966 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d24a53-be32-4edd-805b-99c559d3cba3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:02:14.618246 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d24a53-be32-4edd-805b-99c559d3cba3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:02:14.631051 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d24a53-be32-4edd-805b-99c559d3cba3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:02:14.632605 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d24a53-be32-4edd-805b-99c559d3cba3 service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:02:14.643840 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 737/1455] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:02:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 15:02:17.061222 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e18ac98-cce1-4f87-9238-c6cf4e3a60e6 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:02:17.061808 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e18ac98-cce1-4f87-9238-c6cf4e3a60e6 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:02:17.061808 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e18ac98-cce1-4f87-9238-c6cf4e3a60e6 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:02:17.063189 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 719/1456] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 26 15:02:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 15:02:17.068573 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195e96e4-ce4b-4ed0-a562-482ccb25a47f None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:02:17.069005 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195e96e4-ce4b-4ed0-a562-482ccb25a47f None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:02:17.069106 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195e96e4-ce4b-4ed0-a562-482ccb25a47f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:02:17.087126 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-195e96e4-ce4b-4ed0-a562-482ccb25a47f None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:02:17.088827 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-195e96e4-ce4b-4ed0-a562-482ccb25a47f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:02:17.096553 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 738/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 26 15:02:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:02:17.134848 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e18ac98-cce1-4f87-9238-c6cf4e3a60e6 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:02:17.148974 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e18ac98-cce1-4f87-9238-c6cf4e3a60e6 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:02:17.153094 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284f2638-f528-4376-a373-57c2d45bc096 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0156a2f6f3794f479e76b85bdbbe5c26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:02:17.153768 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284f2638-f528-4376-a373-57c2d45bc096 service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:02:17.154053 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284f2638-f528-4376-a373-57c2d45bc096 service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:02:17.154331 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284f2638-f528-4376-a373-57c2d45bc096 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:02:17.163306 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284f2638-f528-4376-a373-57c2d45bc096 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:02:17.164745 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284f2638-f528-4376-a373-57c2d45bc096 service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:02:17.176986 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 720/1458] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:02:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 15:02:19.509556 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15876e3c-ebed-4b63-b78f-8caebbd8cdb6 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:02:19.510685 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15876e3c-ebed-4b63-b78f-8caebbd8cdb6 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:02:19.510685 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15876e3c-ebed-4b63-b78f-8caebbd8cdb6 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:02:19.512045 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 739/1459] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 26 15:02:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 15:02:19.517407 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b54c73-31f6-4d74-859a-eb2264630d20 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:02:19.517873 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b54c73-31f6-4d74-859a-eb2264630d20 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:02:19.518012 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b54c73-31f6-4d74-859a-eb2264630d20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:02:19.523967 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=70 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:02:19.536790 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-16b54c73-31f6-4d74-859a-eb2264630d20 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:02:19.538009 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-16b54c73-31f6-4d74-859a-eb2264630d20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:02:19.545994 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 721/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 26 15:02:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:02:19.585086 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15876e3c-ebed-4b63-b78f-8caebbd8cdb6 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:02:19.590491 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=73 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:02:19.598095 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15876e3c-ebed-4b63-b78f-8caebbd8cdb6 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:02:19.601735 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24bbdea-d67c-4501-80da-d591de281a04 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0156a2f6f3794f479e76b85bdbbe5c26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:02:19.602384 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24bbdea-d67c-4501-80da-d591de281a04 service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:02:19.602992 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24bbdea-d67c-4501-80da-d591de281a04 service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:02:19.602992 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24bbdea-d67c-4501-80da-d591de281a04 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:02:19.612601 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24bbdea-d67c-4501-80da-d591de281a04 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:02:19.614059 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24bbdea-d67c-4501-80da-d591de281a04 service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:02:19.624085 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 740/1461] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:02:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 15:02:21.882905 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d1018c-59e7-4bb8-8b6c-be79287908b4 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:02:21.883460 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d1018c-59e7-4bb8-8b6c-be79287908b4 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:02:21.883460 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d1018c-59e7-4bb8-8b6c-be79287908b4 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:02:21.884772 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 722/1462] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 26 15:02:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 15:02:21.890106 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00964d66-7e25-424e-a2b0-ac311c22a5d8 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:02:21.890522 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00964d66-7e25-424e-a2b0-ac311c22a5d8 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:02:21.890663 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00964d66-7e25-424e-a2b0-ac311c22a5d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:02:21.909944 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-00964d66-7e25-424e-a2b0-ac311c22a5d8 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:02:21.910654 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-00964d66-7e25-424e-a2b0-ac311c22a5d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:02:21.921712 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 741/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 26 15:02:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:02:21.958429 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d1018c-59e7-4bb8-8b6c-be79287908b4 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:02:21.969601 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d1018c-59e7-4bb8-8b6c-be79287908b4 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:02:21.973073 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29a5615-15f4-419b-b5e8-e7d629ebd0fb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0156a2f6f3794f479e76b85bdbbe5c26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:02:21.973741 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29a5615-15f4-419b-b5e8-e7d629ebd0fb service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:02:21.974028 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29a5615-15f4-419b-b5e8-e7d629ebd0fb service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:02:21.974304 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29a5615-15f4-419b-b5e8-e7d629ebd0fb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:02:21.983399 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29a5615-15f4-419b-b5e8-e7d629ebd0fb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:02:21.985043 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29a5615-15f4-419b-b5e8-e7d629ebd0fb service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:02:21.997464 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 723/1464] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:02:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 15:02:24.238072 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fff7a63-2a65-4be6-9b08-f60adf4a2c1d None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:02:24.238664 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fff7a63-2a65-4be6-9b08-f60adf4a2c1d None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:02:24.238664 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fff7a63-2a65-4be6-9b08-f60adf4a2c1d None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:02:24.240096 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 742/1465] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 26 15:02:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 15:02:24.245559 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48af88ab-2b7d-42ed-9662-fb086ee3983a None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:02:24.245972 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48af88ab-2b7d-42ed-9662-fb086ee3983a None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:02:24.246138 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48af88ab-2b7d-42ed-9662-fb086ee3983a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:02:24.264615 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-48af88ab-2b7d-42ed-9662-fb086ee3983a None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:02:24.265846 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-48af88ab-2b7d-42ed-9662-fb086ee3983a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:02:24.273748 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 724/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 26 15:02:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:02:24.309627 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fff7a63-2a65-4be6-9b08-f60adf4a2c1d None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:02:24.320522 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fff7a63-2a65-4be6-9b08-f60adf4a2c1d None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:02:24.324561 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2d5572-6813-4542-8bf0-8415c3da1672 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0156a2f6f3794f479e76b85bdbbe5c26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:02:24.325197 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2d5572-6813-4542-8bf0-8415c3da1672 service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:02:24.325580 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2d5572-6813-4542-8bf0-8415c3da1672 service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:02:24.325862 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2d5572-6813-4542-8bf0-8415c3da1672 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:02:24.334907 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2d5572-6813-4542-8bf0-8415c3da1672 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:02:24.336349 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2d5572-6813-4542-8bf0-8415c3da1672 service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:02:24.346319 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 743/1467] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:02:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 15:02:26.580269 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb9a94a-dc2e-4391-9b7f-9b201b2ced07 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:02:26.581105 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb9a94a-dc2e-4391-9b7f-9b201b2ced07 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:02:26.581105 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb9a94a-dc2e-4391-9b7f-9b201b2ced07 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:02:26.582210 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 725/1468] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 26 15:02:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 15:02:26.587930 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f5536e-9551-45b5-8238-07c8f75c8b43 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:02:26.588389 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f5536e-9551-45b5-8238-07c8f75c8b43 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:02:26.588498 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f5536e-9551-45b5-8238-07c8f75c8b43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:02:26.606850 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-18f5536e-9551-45b5-8238-07c8f75c8b43 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:02:26.608067 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-18f5536e-9551-45b5-8238-07c8f75c8b43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:02:26.620041 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 744/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 26 15:02:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:02:26.656049 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb9a94a-dc2e-4391-9b7f-9b201b2ced07 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:02:26.668709 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb9a94a-dc2e-4391-9b7f-9b201b2ced07 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:02:26.672697 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61c36453-4be4-42ce-bbdb-e3377e774d41 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0156a2f6f3794f479e76b85bdbbe5c26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:02:26.673378 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c36453-4be4-42ce-bbdb-e3377e774d41 service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:02:26.673706 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c36453-4be4-42ce-bbdb-e3377e774d41 service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:02:26.673981 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c36453-4be4-42ce-bbdb-e3377e774d41 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:02:26.683933 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c36453-4be4-42ce-bbdb-e3377e774d41 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:02:26.685365 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61c36453-4be4-42ce-bbdb-e3377e774d41 service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:02:26.694870 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 726/1470] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:02:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 15:02:36.600985 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=17 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:02:36.690645 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=17 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:03:00.245244 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68358dd6-f123-4a4c-b772-672051d2d81e None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:03:00.245821 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68358dd6-f123-4a4c-b772-672051d2d81e None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:03:00.245821 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68358dd6-f123-4a4c-b772-672051d2d81e None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:03:00.247225 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 745/1471] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 15:03:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 15:03:00.252747 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c0ca61-7885-4cd1-be51-94da9812cf2b None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:03:00.253204 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c0ca61-7885-4cd1-be51-94da9812cf2b None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:03:00.253319 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c0ca61-7885-4cd1-be51-94da9812cf2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:03:00.271787 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-c3c0ca61-7885-4cd1-be51-94da9812cf2b None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:03:00.272978 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-c3c0ca61-7885-4cd1-be51-94da9812cf2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:03:00.281440 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 727/1472] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 15:03:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:03:00.329144 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13397e62-82cf-43c7-b0cc-3158e4d49047 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:03:00.329432 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13397e62-82cf-43c7-b0cc-3158e4d49047 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:03:00.329749 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13397e62-82cf-43c7-b0cc-3158e4d49047 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:03:00.331102 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 746/1473] 10.222.0.46 () {54 vars in 962 bytes} [Thu May 26 15:03:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 15:03:00.336155 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ff9900-e566-4580-8d91-30de53db652a None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:03:00.336447 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ff9900-e566-4580-8d91-30de53db652a None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:03:00.336728 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ff9900-e566-4580-8d91-30de53db652a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:03:00.355727 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-42ff9900-e566-4580-8d91-30de53db652a None None] MFA Rules not processed for user `0a436dbdee624f44ad76cca8416b0ad1`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:03:00.356952 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-42ff9900-e566-4580-8d91-30de53db652a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:03:00.363710 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 728/1474] 10.222.0.46 () {58 vars in 1059 bytes} [Thu May 26 15:03:00 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:03:00.371076 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13397e62-82cf-43c7-b0cc-3158e4d49047 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:03:00.382402 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13397e62-82cf-43c7-b0cc-3158e4d49047 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:03:00.386776 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0745dcc-f86c-4f3b-a45c-efcf03de884b service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a436dbdee624f44ad76cca8416b0ad1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:03:00.387415 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0745dcc-f86c-4f3b-a45c-efcf03de884b service glance] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:03:00.387702 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0745dcc-f86c-4f3b-a45c-efcf03de884b service glance] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:03:00.387976 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0745dcc-f86c-4f3b-a45c-efcf03de884b service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:03:00.398705 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0745dcc-f86c-4f3b-a45c-efcf03de884b service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:03:00.400901 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0745dcc-f86c-4f3b-a45c-efcf03de884b service glance] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:03:00.410558 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 747/1475] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:03:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 15:03:02.776473 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d4a93a-b2a5-47a5-894a-fc86641b04f0 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:03:02.777010 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d4a93a-b2a5-47a5-894a-fc86641b04f0 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:03:02.777010 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d4a93a-b2a5-47a5-894a-fc86641b04f0 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:03:02.779217 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 729/1476] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 15:03:02 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 15:03:02.785516 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b874f28-0dba-44e6-bd06-83bf91903328 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:03:02.786753 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b874f28-0dba-44e6-bd06-83bf91903328 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:03:02.786753 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b874f28-0dba-44e6-bd06-83bf91903328 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:03:02.804737 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-8b874f28-0dba-44e6-bd06-83bf91903328 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:03:02.806633 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-8b874f28-0dba-44e6-bd06-83bf91903328 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:03:02.817661 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 748/1477] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 15:03:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:03:02.841884 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d4a93a-b2a5-47a5-894a-fc86641b04f0 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:03:02.854824 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d4a93a-b2a5-47a5-894a-fc86641b04f0 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:03:02.859183 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1bb1f3-9b3e-41bb-a614-8e5788f24afd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:03:02.859820 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1bb1f3-9b3e-41bb-a614-8e5788f24afd admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:03:02.860134 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1bb1f3-9b3e-41bb-a614-8e5788f24afd admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:03:02.860407 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1bb1f3-9b3e-41bb-a614-8e5788f24afd admin admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:03:02.861126 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1bb1f3-9b3e-41bb-a614-8e5788f24afd admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:03:02.862900 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: WARNING py.warnings [None req-6c1bb1f3-9b3e-41bb-a614-8e5788f24afd admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 26 15:03:02.862900 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: warnings.warn(msg) May 26 15:03:02.862900 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: May 26 15:03:02.864036 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1bb1f3-9b3e-41bb-a614-8e5788f24afd admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:03:02.882936 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 730/1478] 10.222.0.46 () {56 vars in 1108 bytes} [Thu May 26 15:03:02 2022] GET /identity/v3/projects => generated 2553 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 15:03:05.134296 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c351a791-70d2-4a4b-8723-e50718b578e0 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:03:05.134876 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c351a791-70d2-4a4b-8723-e50718b578e0 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:03:05.134876 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c351a791-70d2-4a4b-8723-e50718b578e0 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:03:05.136220 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 749/1479] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 15:03:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 15:03:05.141633 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99893401-3821-4160-ab23-e0c17afb1ea6 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:03:05.142110 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99893401-3821-4160-ab23-e0c17afb1ea6 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:03:05.142225 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99893401-3821-4160-ab23-e0c17afb1ea6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:03:05.160483 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-99893401-3821-4160-ab23-e0c17afb1ea6 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:03:05.161743 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-99893401-3821-4160-ab23-e0c17afb1ea6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:03:05.170551 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 731/1480] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 15:03:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:03:05.213731 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c351a791-70d2-4a4b-8723-e50718b578e0 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:03:05.228459 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c351a791-70d2-4a4b-8723-e50718b578e0 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:03:05.233836 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ccb388-4580-40ee-9add-e43a468981a8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6639353c70449d1b3d10cd6487c3b00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:03:05.234468 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ccb388-4580-40ee-9add-e43a468981a8 service nova] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:03:05.234757 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ccb388-4580-40ee-9add-e43a468981a8 service nova] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:03:05.235041 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ccb388-4580-40ee-9add-e43a468981a8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:03:05.243097 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ccb388-4580-40ee-9add-e43a468981a8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:03:05.245149 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ccb388-4580-40ee-9add-e43a468981a8 service nova] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:03:05.254813 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 750/1481] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:03:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 15:03:07.685026 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61f5fce-c451-4158-825f-79b6e92e2ade None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:03:07.685775 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61f5fce-c451-4158-825f-79b6e92e2ade None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:03:07.685775 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61f5fce-c451-4158-825f-79b6e92e2ade None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:03:07.687004 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 732/1482] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 15:03:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 15:03:07.692514 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a0f003-0239-4531-8685-bcb5c6b0232a None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:03:07.692911 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a0f003-0239-4531-8685-bcb5c6b0232a None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:03:07.693064 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a0f003-0239-4531-8685-bcb5c6b0232a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:03:07.711102 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-b7a0f003-0239-4531-8685-bcb5c6b0232a None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:03:07.712915 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-b7a0f003-0239-4531-8685-bcb5c6b0232a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:03:07.720820 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 751/1483] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 15:03:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:03:07.756590 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61f5fce-c451-4158-825f-79b6e92e2ade None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:03:07.768746 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61f5fce-c451-4158-825f-79b6e92e2ade None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:03:07.772264 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bacc37-ea0a-484d-b168-02aa84b7a965 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0156a2f6f3794f479e76b85bdbbe5c26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:03:07.774122 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bacc37-ea0a-484d-b168-02aa84b7a965 service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:03:07.775062 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bacc37-ea0a-484d-b168-02aa84b7a965 service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:03:07.775344 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bacc37-ea0a-484d-b168-02aa84b7a965 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:03:07.785547 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29bacc37-ea0a-484d-b168-02aa84b7a965 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:03:07.787656 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29bacc37-ea0a-484d-b168-02aa84b7a965 service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:03:07.798850 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 733/1484] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:03:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 15:03:10.083570 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb54fc07-00c5-47a3-9b93-3a5c783e04b0 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:03:10.084166 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb54fc07-00c5-47a3-9b93-3a5c783e04b0 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:03:10.084166 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb54fc07-00c5-47a3-9b93-3a5c783e04b0 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:03:10.085644 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 752/1485] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 15:03:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 15:03:10.091568 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993dec84-e0e2-421e-a40b-a3a6aadd8d39 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:03:10.092040 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993dec84-e0e2-421e-a40b-a3a6aadd8d39 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:03:10.092140 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993dec84-e0e2-421e-a40b-a3a6aadd8d39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:03:10.109867 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-993dec84-e0e2-421e-a40b-a3a6aadd8d39 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:03:10.111115 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-993dec84-e0e2-421e-a40b-a3a6aadd8d39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:03:10.118998 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 734/1486] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 15:03:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:03:10.158547 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb54fc07-00c5-47a3-9b93-3a5c783e04b0 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:03:10.169824 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb54fc07-00c5-47a3-9b93-3a5c783e04b0 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:03:10.173590 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0dc67d-fa9f-4534-915e-260347857337 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0156a2f6f3794f479e76b85bdbbe5c26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:03:10.174228 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0dc67d-fa9f-4534-915e-260347857337 service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:03:10.174511 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0dc67d-fa9f-4534-915e-260347857337 service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:03:10.174797 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0dc67d-fa9f-4534-915e-260347857337 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:03:10.184039 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0dc67d-fa9f-4534-915e-260347857337 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:03:10.185555 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0dc67d-fa9f-4534-915e-260347857337 service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:03:10.194917 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 753/1487] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:03:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 15:03:12.584918 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88561d32-f47f-4d91-8cd2-5236f5de3b99 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:03:12.585604 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88561d32-f47f-4d91-8cd2-5236f5de3b99 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:03:12.585604 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88561d32-f47f-4d91-8cd2-5236f5de3b99 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:03:12.586924 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 735/1488] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 15:03:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 15:03:12.592553 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2578a306-6d8a-46da-a382-8492511404c5 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:03:12.593028 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2578a306-6d8a-46da-a382-8492511404c5 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:03:12.593142 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2578a306-6d8a-46da-a382-8492511404c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:03:12.611401 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-2578a306-6d8a-46da-a382-8492511404c5 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:03:12.612605 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-2578a306-6d8a-46da-a382-8492511404c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:03:12.621268 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 754/1489] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 15:03:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:03:12.662239 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88561d32-f47f-4d91-8cd2-5236f5de3b99 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:03:12.674319 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88561d32-f47f-4d91-8cd2-5236f5de3b99 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:03:12.678091 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3a3100-aace-4400-92c9-c184ef18164e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0156a2f6f3794f479e76b85bdbbe5c26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:03:12.678735 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3a3100-aace-4400-92c9-c184ef18164e service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:03:12.679009 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3a3100-aace-4400-92c9-c184ef18164e service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:03:12.679281 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3a3100-aace-4400-92c9-c184ef18164e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:03:12.688528 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3a3100-aace-4400-92c9-c184ef18164e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:03:12.689995 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3a3100-aace-4400-92c9-c184ef18164e service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:03:12.699218 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 736/1490] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:03:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 15:03:15.198597 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec534cf3-9851-4a17-a089-2e6b09fa8aa2 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:03:15.199183 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec534cf3-9851-4a17-a089-2e6b09fa8aa2 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:03:15.199183 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec534cf3-9851-4a17-a089-2e6b09fa8aa2 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:03:15.200627 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 755/1491] 10.222.0.46 () {54 vars in 926 bytes} [Thu May 26 15:03:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 15:03:15.206675 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9e95f2-f7df-4041-be71-05e8cefc5906 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:03:15.207204 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9e95f2-f7df-4041-be71-05e8cefc5906 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:03:15.207329 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9e95f2-f7df-4041-be71-05e8cefc5906 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:03:15.225619 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-5d9e95f2-f7df-4041-be71-05e8cefc5906 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:03:15.226834 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-5d9e95f2-f7df-4041-be71-05e8cefc5906 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:03:15.234794 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 737/1492] 10.222.0.46 () {58 vars in 1023 bytes} [Thu May 26 15:03:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:03:15.274166 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec534cf3-9851-4a17-a089-2e6b09fa8aa2 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:03:15.285583 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec534cf3-9851-4a17-a089-2e6b09fa8aa2 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:03:15.289245 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3980ff0f-c2f1-439b-9616-2ddc2245ec1c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0156a2f6f3794f479e76b85bdbbe5c26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:03:15.289924 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3980ff0f-c2f1-439b-9616-2ddc2245ec1c service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:03:15.290205 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3980ff0f-c2f1-439b-9616-2ddc2245ec1c service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:03:15.290482 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3980ff0f-c2f1-439b-9616-2ddc2245ec1c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:03:15.300113 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3980ff0f-c2f1-439b-9616-2ddc2245ec1c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:03:15.301719 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3980ff0f-c2f1-439b-9616-2ddc2245ec1c service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:03:15.311796 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 756/1493] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:03:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 15:03:25.220181 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=33 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:03:25.309341 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=33 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:04:56.863872 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f3953e-6957-4cd9-ac84-68d005511963 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:04:56.864482 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f3953e-6957-4cd9-ac84-68d005511963 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:04:56.864482 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f3953e-6957-4cd9-ac84-68d005511963 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:04:56.883781 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-98f3953e-6957-4cd9-ac84-68d005511963 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:04:56.884922 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-98f3953e-6957-4cd9-ac84-68d005511963 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:04:56.895010 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 738/1494] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:04:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 15:04:56.905083 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8417578d-c92f-446f-8a4c-cc34a4c8d453 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:04:56.905883 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8417578d-c92f-446f-8a4c-cc34a4c8d453 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:04:56.905883 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8417578d-c92f-446f-8a4c-cc34a4c8d453 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:04:56.924144 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-8417578d-c92f-446f-8a4c-cc34a4c8d453 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:04:56.925361 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-8417578d-c92f-446f-8a4c-cc34a4c8d453 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:04:56.933354 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 757/1495] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:04:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:04:56.938234 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f3953e-6957-4cd9-ac84-68d005511963 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:04:56.948585 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f3953e-6957-4cd9-ac84-68d005511963 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:04:56.953127 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6b2ba5-1c85-4df3-a9d7-62e249e20d96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:04:56.953827 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6b2ba5-1c85-4df3-a9d7-62e249e20d96 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:04:56.954109 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6b2ba5-1c85-4df3-a9d7-62e249e20d96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:04:56.954387 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6b2ba5-1c85-4df3-a9d7-62e249e20d96 admin admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:04:56.955120 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6b2ba5-1c85-4df3-a9d7-62e249e20d96 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 15:04:56.955427 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6b2ba5-1c85-4df3-a9d7-62e249e20d96 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:04:56.956483 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: WARNING py.warnings [None req-4e6b2ba5-1c85-4df3-a9d7-62e249e20d96 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 26 15:04:56.956483 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: warnings.warn(msg) May 26 15:04:56.956483 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: May 26 15:04:56.957377 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6b2ba5-1c85-4df3-a9d7-62e249e20d96 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:04:56.970496 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 739/1496] 10.222.0.46 () {58 vars in 1152 bytes} [Thu May 26 15:04:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 15:04:56.979711 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8417578d-c92f-446f-8a4c-cc34a4c8d453 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:04:56.989008 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8417578d-c92f-446f-8a4c-cc34a4c8d453 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:04:56.993768 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634472b7-db0d-40ff-8ffe-8b549fb29b78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:04:56.994428 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634472b7-db0d-40ff-8ffe-8b549fb29b78 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:04:56.994716 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634472b7-db0d-40ff-8ffe-8b549fb29b78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:04:56.994998 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634472b7-db0d-40ff-8ffe-8b549fb29b78 admin admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:04:56.995697 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634472b7-db0d-40ff-8ffe-8b549fb29b78 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:04:56.997533 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: WARNING py.warnings [None req-634472b7-db0d-40ff-8ffe-8b549fb29b78 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 26 15:04:56.997533 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: warnings.warn(msg) May 26 15:04:56.997533 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: May 26 15:04:56.998758 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634472b7-db0d-40ff-8ffe-8b549fb29b78 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:04:57.037227 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 758/1497] 10.222.0.46 () {60 vars in 1152 bytes} [Thu May 26 15:04:56 2022] POST /identity/v3/projects => generated 377 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 15:04:57.041497 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6b2ba5-1c85-4df3-a9d7-62e249e20d96 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:04:57.051599 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6b2ba5-1c85-4df3-a9d7-62e249e20d96 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:04:57.111572 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0931a960-7ac8-4a4f-a47e-da01623b1cee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:04:57.112247 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0931a960-7ac8-4a4f-a47e-da01623b1cee admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:04:57.112528 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0931a960-7ac8-4a4f-a47e-da01623b1cee admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:04:57.112807 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0931a960-7ac8-4a4f-a47e-da01623b1cee admin admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:04:57.113521 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0931a960-7ac8-4a4f-a47e-da01623b1cee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:04:57.114687 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: WARNING py.warnings [None req-0931a960-7ac8-4a4f-a47e-da01623b1cee admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 26 15:04:57.114687 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: warnings.warn(msg) May 26 15:04:57.114687 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: May 26 15:04:57.115887 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0931a960-7ac8-4a4f-a47e-da01623b1cee admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:04:57.136007 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 740/1498] 10.222.0.46 () {60 vars in 1143 bytes} [Thu May 26 15:04:57 2022] POST /identity/v3/users => generated 289 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 15:04:57.140089 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634472b7-db0d-40ff-8ffe-8b549fb29b78 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:04:57.151080 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634472b7-db0d-40ff-8ffe-8b549fb29b78 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:04:57.156007 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20761a4-d660-4549-9039-009edb78cef3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:04:57.156665 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20761a4-d660-4549-9039-009edb78cef3 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:04:57.156949 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20761a4-d660-4549-9039-009edb78cef3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:04:57.157227 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20761a4-d660-4549-9039-009edb78cef3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:04:57.157990 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20761a4-d660-4549-9039-009edb78cef3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:04:57.159025 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: WARNING py.warnings [None req-e20761a4-d660-4549-9039-009edb78cef3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 26 15:04:57.159025 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: warnings.warn(msg) May 26 15:04:57.159025 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: May 26 15:04:57.159906 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20761a4-d660-4549-9039-009edb78cef3 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:04:57.176684 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 759/1499] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 15:04:57 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 15:04:57.181084 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0931a960-7ac8-4a4f-a47e-da01623b1cee admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:04:57.191068 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0931a960-7ac8-4a4f-a47e-da01623b1cee admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:04:57.196391 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9ca637-63d6-4eff-9e4c-51b2b5f3f6c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:04:57.197058 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9ca637-63d6-4eff-9e4c-51b2b5f3f6c9 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:04:57.197347 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9ca637-63d6-4eff-9e4c-51b2b5f3f6c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:04:57.197658 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9ca637-63d6-4eff-9e4c-51b2b5f3f6c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:04:57.198394 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9ca637-63d6-4eff-9e4c-51b2b5f3f6c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:04:57.199424 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: WARNING py.warnings [None req-3e9ca637-63d6-4eff-9e4c-51b2b5f3f6c9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 26 15:04:57.199424 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: warnings.warn(msg) May 26 15:04:57.199424 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: May 26 15:04:57.200307 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9ca637-63d6-4eff-9e4c-51b2b5f3f6c9 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:04:57.215086 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 741/1500] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 15:04:57 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 15:04:57.219538 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20761a4-d660-4549-9039-009edb78cef3 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:04:57.229810 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20761a4-d660-4549-9039-009edb78cef3 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:04:57.234326 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8cf102-0de4-4e2e-8d3c-f26943368cb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:04:57.235205 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8cf102-0de4-4e2e-8d3c-f26943368cb2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:04:57.235485 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8cf102-0de4-4e2e-8d3c-f26943368cb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:04:57.235757 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8cf102-0de4-4e2e-8d3c-f26943368cb2 admin admin] PATH_INFO: `/v3/projects/c231bebb1fe14ee29b28525024414c6e/users/cc1f7b24a63347d69be2a4c0c07268bf/roles/d4088d1845a24fbd9bd8975ce0b1d3d0` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:04:57.251147 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8cf102-0de4-4e2e-8d3c-f26943368cb2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c231bebb1fe14ee29b28525024414c6e, user_id=cc1f7b24a63347d69be2a4c0c07268bf, role_id=d4088d1845a24fbd9bd8975ce0b1d3d0)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:04:57.252455 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: WARNING py.warnings [None req-8a8cf102-0de4-4e2e-8d3c-f26943368cb2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 26 15:04:57.252455 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: warnings.warn(msg) May 26 15:04:57.252455 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: May 26 15:04:57.254501 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8cf102-0de4-4e2e-8d3c-f26943368cb2 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:04:57.271191 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 760/1501] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 15:04:57 2022] PUT /identity/v3/projects/c231bebb1fe14ee29b28525024414c6e/users/cc1f7b24a63347d69be2a4c0c07268bf/roles/d4088d1845a24fbd9bd8975ce0b1d3d0 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:04:57.282798 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074ca583-e913-4259-a9b7-cd49d49b5ec8 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:04:57.283098 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074ca583-e913-4259-a9b7-cd49d49b5ec8 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:04:57.283403 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074ca583-e913-4259-a9b7-cd49d49b5ec8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:04:57.310239 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-074ca583-e913-4259-a9b7-cd49d49b5ec8 None None] MFA Rules not processed for user `cc1f7b24a63347d69be2a4c0c07268bf`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:04:57.311472 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-074ca583-e913-4259-a9b7-cd49d49b5ec8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:04:57.382913 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 742/1502] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:04:57 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:04:57.397060 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939bad1f-d348-4e0f-a770-2ebedcf89e84 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:04:57.397352 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939bad1f-d348-4e0f-a770-2ebedcf89e84 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:04:57.397660 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939bad1f-d348-4e0f-a770-2ebedcf89e84 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:04:57.399042 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 761/1503] 10.222.0.46 () {54 vars in 963 bytes} [Thu May 26 15:04:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 15:04:57.404292 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb52bdea-06f3-4cbd-98dc-48801d92f17f None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:04:57.404622 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb52bdea-06f3-4cbd-98dc-48801d92f17f None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:04:57.404906 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb52bdea-06f3-4cbd-98dc-48801d92f17f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:04:57.423875 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-fb52bdea-06f3-4cbd-98dc-48801d92f17f None None] MFA Rules not processed for user `d5b5b35614a547d783e6e77ddb8c1c03`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:04:57.425167 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-fb52bdea-06f3-4cbd-98dc-48801d92f17f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:04:57.469533 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:04:57.478390 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 743/1504] 10.222.0.46 () {58 vars in 1060 bytes} [Thu May 26 15:04:57 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:04:57.486277 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939bad1f-d348-4e0f-a770-2ebedcf89e84 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:04:57.497325 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939bad1f-d348-4e0f-a770-2ebedcf89e84 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:04:57.503137 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b2638b-1b32-4610-9b45-4dc7cb9269f6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5b5b35614a547d783e6e77ddb8c1c03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:04:57.503787 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b2638b-1b32-4610-9b45-4dc7cb9269f6 service cinder] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:04:57.504074 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b2638b-1b32-4610-9b45-4dc7cb9269f6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:04:57.504356 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b2638b-1b32-4610-9b45-4dc7cb9269f6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:04:57.513762 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b2638b-1b32-4610-9b45-4dc7cb9269f6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:04:57.515200 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b2638b-1b32-4610-9b45-4dc7cb9269f6 service cinder] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:04:57.524880 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 762/1505] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:04:57 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 15:04:57.573838 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2150e6ee-da7e-438e-ae86-c587c6c33e7c None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:04:57.573838 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2150e6ee-da7e-438e-ae86-c587c6c33e7c None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:04:57.573838 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2150e6ee-da7e-438e-ae86-c587c6c33e7c None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:04:57.574255 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 744/1506] 10.222.0.46 () {52 vars in 829 bytes} [Thu May 26 15:04:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 15:04:57.578083 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b2638b-1b32-4610-9b45-4dc7cb9269f6 service cinder] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:04:57.641921 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b2638b-1b32-4610-9b45-4dc7cb9269f6 service cinder] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:04:57.646875 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a843c142-7a33-4c0a-b827-d1727176a4ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:04:57.647753 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a843c142-7a33-4c0a-b827-d1727176a4ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:04:57.648039 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a843c142-7a33-4c0a-b827-d1727176a4ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:04:57.648316 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a843c142-7a33-4c0a-b827-d1727176a4ac admin admin] PATH_INFO: `/v3/users/cc1f7b24a63347d69be2a4c0c07268bf` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:04:57.649545 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a843c142-7a33-4c0a-b827-d1727176a4ac admin admin] RBAC: Authorizing `identity:delete_user(user_id=cc1f7b24a63347d69be2a4c0c07268bf)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:04:57.650566 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: WARNING py.warnings [None req-a843c142-7a33-4c0a-b827-d1727176a4ac admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 26 15:04:57.650566 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: warnings.warn(msg) May 26 15:04:57.650566 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: May 26 15:04:57.651788 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a843c142-7a33-4c0a-b827-d1727176a4ac admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:04:57.723226 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-a843c142-7a33-4c0a-b827-d1727176a4ac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cc1f7b24a63347d69be2a4c0c07268bf'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:04:57.735782 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-a843c142-7a33-4c0a-b827-d1727176a4ac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cc1f7b24a63347d69be2a4c0c07268bf'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:04:57.736434 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-a843c142-7a33-4c0a-b827-d1727176a4ac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cc1f7b24a63347d69be2a4c0c07268bf'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:04:57.756386 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-a843c142-7a33-4c0a-b827-d1727176a4ac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cc1f7b24a63347d69be2a4c0c07268bf'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:04:57.765178 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:04:57.770196 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-a843c142-7a33-4c0a-b827-d1727176a4ac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cc1f7b24a63347d69be2a4c0c07268bf'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:04:57.778156 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 763/1507] 10.222.0.46 () {58 vars in 1223 bytes} [Thu May 26 15:04:57 2022] DELETE /identity/v3/users/cc1f7b24a63347d69be2a4c0c07268bf => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:04:57.782626 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9c9fbf-fb44-49f1-a7e1-a331a294b1f3 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:04:57.782940 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9c9fbf-fb44-49f1-a7e1-a331a294b1f3 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:04:57.783220 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9c9fbf-fb44-49f1-a7e1-a331a294b1f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:04:57.801222 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-3f9c9fbf-fb44-49f1-a7e1-a331a294b1f3 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:04:57.803094 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-3f9c9fbf-fb44-49f1-a7e1-a331a294b1f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:04:57.867421 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 745/1508] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:04:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:04:57.877525 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a843c142-7a33-4c0a-b827-d1727176a4ac admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:04:57.878948 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-a843c142-7a33-4c0a-b827-d1727176a4ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:04:57.938841 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a843c142-7a33-4c0a-b827-d1727176a4ac admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:04:57.943129 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2769ac53-117f-45a5-8345-f79355992243 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0156a2f6f3794f479e76b85bdbbe5c26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:04:57.943781 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2769ac53-117f-45a5-8345-f79355992243 service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:04:57.944061 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2769ac53-117f-45a5-8345-f79355992243 service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:04:57.944334 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2769ac53-117f-45a5-8345-f79355992243 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:04:57.953706 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2769ac53-117f-45a5-8345-f79355992243 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:04:57.955185 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2769ac53-117f-45a5-8345-f79355992243 service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:04:57.964981 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 764/1509] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:04:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 15:04:58.336130 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9c9fbf-fb44-49f1-a7e1-a331a294b1f3 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:04:58.337606 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-3f9c9fbf-fb44-49f1-a7e1-a331a294b1f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:04:58.349783 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9c9fbf-fb44-49f1-a7e1-a331a294b1f3 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:04:58.354594 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8514a9a-b423-4863-af6e-c868f5fd96bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:04:58.355478 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8514a9a-b423-4863-af6e-c868f5fd96bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:04:58.355755 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8514a9a-b423-4863-af6e-c868f5fd96bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:04:58.356030 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8514a9a-b423-4863-af6e-c868f5fd96bb admin admin] PATH_INFO: `/v3/projects/c231bebb1fe14ee29b28525024414c6e` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:04:58.357415 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8514a9a-b423-4863-af6e-c868f5fd96bb admin admin] RBAC: Authorizing `identity:delete_project(project_id=c231bebb1fe14ee29b28525024414c6e)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:04:58.358460 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: WARNING py.warnings [None req-f8514a9a-b423-4863-af6e-c868f5fd96bb admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 26 15:04:58.358460 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: warnings.warn(msg) May 26 15:04:58.358460 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: May 26 15:04:58.359793 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8514a9a-b423-4863-af6e-c868f5fd96bb admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:04:58.424967 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-f8514a9a-b423-4863-af6e-c868f5fd96bb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c231bebb1fe14ee29b28525024414c6e', 'request_id': 'req-f8514a9a-b423-4863-af6e-c868f5fd96bb'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:04:58.438400 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-f8514a9a-b423-4863-af6e-c868f5fd96bb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c231bebb1fe14ee29b28525024414c6e', 'request_id': 'req-f8514a9a-b423-4863-af6e-c868f5fd96bb'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:04:58.444551 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-f8514a9a-b423-4863-af6e-c868f5fd96bb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c231bebb1fe14ee29b28525024414c6e', 'request_id': 'req-f8514a9a-b423-4863-af6e-c868f5fd96bb'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:04:58.461414 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-f8514a9a-b423-4863-af6e-c868f5fd96bb admin admin] The token cache is being invalidate because project c231bebb1fe14ee29b28525024414c6e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 15:04:58.461666 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-f8514a9a-b423-4863-af6e-c868f5fd96bb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:04:58.462305 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-f8514a9a-b423-4863-af6e-c868f5fd96bb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:04:58.463987 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 746/1510] 10.222.0.46 () {58 vars in 1232 bytes} [Thu May 26 15:04:58 2022] DELETE /identity/v3/projects/c231bebb1fe14ee29b28525024414c6e => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:05:07.960706 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=29 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:05:08.457525 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=55,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:06:42.246566 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb370cb-6da8-49f3-acb3-c906f792c87c None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:06:42.247172 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb370cb-6da8-49f3-acb3-c906f792c87c None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:06:42.247172 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb370cb-6da8-49f3-acb3-c906f792c87c None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:06:42.248513 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 765/1511] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 26 15:06:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 15:06:42.253852 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d53485b-67fe-4aab-b2ea-410d9353bf18 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:06:42.254310 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d53485b-67fe-4aab-b2ea-410d9353bf18 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:06:42.254417 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d53485b-67fe-4aab-b2ea-410d9353bf18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:06:42.273734 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-0d53485b-67fe-4aab-b2ea-410d9353bf18 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:06:42.275000 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-0d53485b-67fe-4aab-b2ea-410d9353bf18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:06:42.333028 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 747/1512] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 26 15:06:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:06:42.504761 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb370cb-6da8-49f3-acb3-c906f792c87c None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:06:42.506607 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-8bb370cb-6da8-49f3-acb3-c906f792c87c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:06:42.583586 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb370cb-6da8-49f3-acb3-c906f792c87c None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:06:42.589090 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aedafe9-9193-4fb8-aada-4fbab8973b8d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6639353c70449d1b3d10cd6487c3b00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:06:42.589767 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aedafe9-9193-4fb8-aada-4fbab8973b8d service nova] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:06:42.590048 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aedafe9-9193-4fb8-aada-4fbab8973b8d service nova] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:06:42.590323 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aedafe9-9193-4fb8-aada-4fbab8973b8d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:06:42.599440 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aedafe9-9193-4fb8-aada-4fbab8973b8d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:06:42.601036 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aedafe9-9193-4fb8-aada-4fbab8973b8d service nova] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:06:42.611496 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 766/1513] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:06:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 15:06:43.501956 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4647a9cd-eade-4edf-9fdd-5b1be6c2ef1b None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:06:43.502664 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4647a9cd-eade-4edf-9fdd-5b1be6c2ef1b None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:06:43.502664 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4647a9cd-eade-4edf-9fdd-5b1be6c2ef1b None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:06:43.504611 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 748/1514] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 26 15:06:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 15:06:43.511720 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3031ec-1328-4ccb-a5da-eea31cba13f9 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:06:43.512214 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3031ec-1328-4ccb-a5da-eea31cba13f9 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:06:43.512320 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3031ec-1328-4ccb-a5da-eea31cba13f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:06:43.531501 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-9c3031ec-1328-4ccb-a5da-eea31cba13f9 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:06:43.532766 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-9c3031ec-1328-4ccb-a5da-eea31cba13f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:06:43.541088 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 767/1515] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 26 15:06:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:06:43.682040 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8f8bca-9382-48ee-b58a-4a7953b43c08 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:06:43.682329 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8f8bca-9382-48ee-b58a-4a7953b43c08 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:06:43.682610 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8f8bca-9382-48ee-b58a-4a7953b43c08 None None] PATH_INFO: `/v3` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:06:43.684009 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 749/1516] 10.222.0.75 () {54 vars in 925 bytes} [Thu May 26 15:06:43 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 15:06:43.691425 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ca5112-986f-4416-87db-85cd0d2c2692 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:06:43.693607 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ca5112-986f-4416-87db-85cd0d2c2692 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:06:43.693607 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ca5112-986f-4416-87db-85cd0d2c2692 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:06:43.700517 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8f8bca-9382-48ee-b58a-4a7953b43c08 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:06:43.713177 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-85ca5112-986f-4416-87db-85cd0d2c2692 None None] MFA Rules not processed for user `bf55a00e3b084aa3b22c25a90e6a773c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:06:43.714535 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-85ca5112-986f-4416-87db-85cd0d2c2692 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:06:43.725240 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8f8bca-9382-48ee-b58a-4a7953b43c08 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:06:43.731672 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b51e00-1b3a-46db-a93b-82040bc9cd59 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6639353c70449d1b3d10cd6487c3b00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:06:43.732677 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b51e00-1b3a-46db-a93b-82040bc9cd59 service nova] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:06:43.732677 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b51e00-1b3a-46db-a93b-82040bc9cd59 service nova] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:06:43.732964 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b51e00-1b3a-46db-a93b-82040bc9cd59 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:06:43.754862 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b51e00-1b3a-46db-a93b-82040bc9cd59 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:06:43.756985 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b51e00-1b3a-46db-a93b-82040bc9cd59 service nova] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:06:43.767951 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 750/1517] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:06:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 15:06:43.801469 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 768/1518] 10.222.0.75 () {58 vars in 1015 bytes} [Thu May 26 15:06:43 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:06:52.656460 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dc0bdf-cce2-473c-b820-1d977ce9cb00 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:06:52.657051 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dc0bdf-cce2-473c-b820-1d977ce9cb00 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:06:52.657174 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dc0bdf-cce2-473c-b820-1d977ce9cb00 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:06:52.658631 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 751/1519] 10.222.0.46 () {54 vars in 960 bytes} [Thu May 26 15:06:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 15:06:52.663733 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8b2d04-a5d3-44af-afd6-285c91a60b71 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:06:52.664235 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8b2d04-a5d3-44af-afd6-285c91a60b71 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:06:52.664395 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8b2d04-a5d3-44af-afd6-285c91a60b71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:06:52.683672 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-cb8b2d04-a5d3-44af-afd6-285c91a60b71 None None] MFA Rules not processed for user `bf55a00e3b084aa3b22c25a90e6a773c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:06:52.684926 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-cb8b2d04-a5d3-44af-afd6-285c91a60b71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:06:52.694207 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 769/1520] 10.222.0.46 () {58 vars in 1057 bytes} [Thu May 26 15:06:52 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:06:52.702852 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92dc0bdf-cce2-473c-b820-1d977ce9cb00 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:06:52.724817 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92dc0bdf-cce2-473c-b820-1d977ce9cb00 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:06:52.733890 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e331049-8188-4647-b7ac-7d2b149d2204 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf55a00e3b084aa3b22c25a90e6a773c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:06:52.734971 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e331049-8188-4647-b7ac-7d2b149d2204 service placement] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:06:52.735443 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e331049-8188-4647-b7ac-7d2b149d2204 service placement] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:06:52.735893 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e331049-8188-4647-b7ac-7d2b149d2204 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:06:52.746868 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e331049-8188-4647-b7ac-7d2b149d2204 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:06:52.748481 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e331049-8188-4647-b7ac-7d2b149d2204 service placement] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:06:52.759933 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 752/1521] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:06:52 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 15:06:55.094788 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de24fbf0-ec91-4539-b65e-8f7c4f4877f4 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:06:55.095370 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de24fbf0-ec91-4539-b65e-8f7c4f4877f4 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:06:55.095370 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de24fbf0-ec91-4539-b65e-8f7c4f4877f4 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:06:55.096755 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 770/1522] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 26 15:06:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 15:06:55.102259 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7ab544-0f48-470c-bf7b-c82bfa8c0bef None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:06:55.102849 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7ab544-0f48-470c-bf7b-c82bfa8c0bef None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:06:55.102979 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7ab544-0f48-470c-bf7b-c82bfa8c0bef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:06:55.121468 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-2c7ab544-0f48-470c-bf7b-c82bfa8c0bef None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:06:55.122911 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-2c7ab544-0f48-470c-bf7b-c82bfa8c0bef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:06:55.131801 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 753/1523] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 26 15:06:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:06:55.286405 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de24fbf0-ec91-4539-b65e-8f7c4f4877f4 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:06:55.304087 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de24fbf0-ec91-4539-b65e-8f7c4f4877f4 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:06:55.312416 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03896b7d-4ed6-4da9-a160-fedd8cd0dbb6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6639353c70449d1b3d10cd6487c3b00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:06:55.313420 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03896b7d-4ed6-4da9-a160-fedd8cd0dbb6 service nova] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:06:55.313763 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03896b7d-4ed6-4da9-a160-fedd8cd0dbb6 service nova] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:06:55.314063 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03896b7d-4ed6-4da9-a160-fedd8cd0dbb6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:06:55.324647 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03896b7d-4ed6-4da9-a160-fedd8cd0dbb6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:06:55.326334 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03896b7d-4ed6-4da9-a160-fedd8cd0dbb6 service nova] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:06:55.336392 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 771/1524] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:06:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 15:06:56.226166 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad9eea7-1a09-41b4-99ce-a7f1e0ddb5e7 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:06:56.226750 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad9eea7-1a09-41b4-99ce-a7f1e0ddb5e7 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:06:56.226750 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad9eea7-1a09-41b4-99ce-a7f1e0ddb5e7 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:06:56.228498 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 754/1525] 127.0.0.1 () {54 vars in 901 bytes} [Thu May 26 15:06:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 15:06:56.233876 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58355157-2752-4b8e-b6d5-1df1c8e65e1e None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:06:56.234345 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58355157-2752-4b8e-b6d5-1df1c8e65e1e None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:06:56.234449 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58355157-2752-4b8e-b6d5-1df1c8e65e1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:06:56.252678 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-58355157-2752-4b8e-b6d5-1df1c8e65e1e None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:06:56.254092 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-58355157-2752-4b8e-b6d5-1df1c8e65e1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:06:56.262430 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 772/1526] 127.0.0.1 () {58 vars in 998 bytes} [Thu May 26 15:06:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 15:06:56.418547 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad9eea7-1a09-41b4-99ce-a7f1e0ddb5e7 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:06:56.433815 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad9eea7-1a09-41b4-99ce-a7f1e0ddb5e7 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:06:56.439224 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab98723-ce07-4766-baa9-f630f0aeb1f7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6639353c70449d1b3d10cd6487c3b00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:06:56.439863 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab98723-ce07-4766-baa9-f630f0aeb1f7 service nova] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:06:56.440145 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab98723-ce07-4766-baa9-f630f0aeb1f7 service nova] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:06:56.440428 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab98723-ce07-4766-baa9-f630f0aeb1f7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:06:56.450240 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab98723-ce07-4766-baa9-f630f0aeb1f7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:06:56.451849 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab98723-ce07-4766-baa9-f630f0aeb1f7 service nova] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:06:56.463148 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 755/1527] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:06:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 15:07:01.054802 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422fb595-f9b5-4b1d-85e2-c7e3932d6311 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:07:01.055551 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422fb595-f9b5-4b1d-85e2-c7e3932d6311 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:07:01.055551 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422fb595-f9b5-4b1d-85e2-c7e3932d6311 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:07:01.056784 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 773/1528] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 26 15:07:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 15:07:01.062356 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79ca6d2-83c8-4729-946d-9024378700cf None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:07:01.063118 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79ca6d2-83c8-4729-946d-9024378700cf None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:07:01.063291 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79ca6d2-83c8-4729-946d-9024378700cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:07:01.082209 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-c79ca6d2-83c8-4729-946d-9024378700cf None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:07:01.083399 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-c79ca6d2-83c8-4729-946d-9024378700cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:07:01.092918 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 756/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 26 15:07:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:07:01.143937 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422fb595-f9b5-4b1d-85e2-c7e3932d6311 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:07:01.145683 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-422fb595-f9b5-4b1d-85e2-c7e3932d6311 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:07:01.216642 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:07:01.222968 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422fb595-f9b5-4b1d-85e2-c7e3932d6311 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:07:01.227335 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1420414-8e2f-45ba-b1dc-7555a831eeea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0156a2f6f3794f479e76b85bdbbe5c26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:07:01.228058 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1420414-8e2f-45ba-b1dc-7555a831eeea service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:07:01.230433 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1420414-8e2f-45ba-b1dc-7555a831eeea service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:07:01.230916 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1420414-8e2f-45ba-b1dc-7555a831eeea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:07:01.243908 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1420414-8e2f-45ba-b1dc-7555a831eeea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:07:01.245652 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1420414-8e2f-45ba-b1dc-7555a831eeea service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:07:01.257600 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 774/1530] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:07:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 15:07:03.580646 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d0e7c2-63fb-4485-9092-05eea2ac164f None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:07:03.582057 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d0e7c2-63fb-4485-9092-05eea2ac164f None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:07:03.582057 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d0e7c2-63fb-4485-9092-05eea2ac164f None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:07:03.583709 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 757/1531] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 26 15:07:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 15:07:03.589871 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a3dfa7-7995-467b-b9fc-0f3e08cc7ad6 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:07:03.590526 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a3dfa7-7995-467b-b9fc-0f3e08cc7ad6 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:07:03.590526 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a3dfa7-7995-467b-b9fc-0f3e08cc7ad6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:07:03.612379 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-72a3dfa7-7995-467b-b9fc-0f3e08cc7ad6 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:07:03.613761 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-72a3dfa7-7995-467b-b9fc-0f3e08cc7ad6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:07:03.622308 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 775/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 26 15:07:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:07:03.664312 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d0e7c2-63fb-4485-9092-05eea2ac164f None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:07:03.677766 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d0e7c2-63fb-4485-9092-05eea2ac164f None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:07:03.683849 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134d92fd-7766-457e-bf67-cd133c24ff6f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0156a2f6f3794f479e76b85bdbbe5c26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:07:03.684564 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134d92fd-7766-457e-bf67-cd133c24ff6f service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:07:03.684961 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134d92fd-7766-457e-bf67-cd133c24ff6f service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:07:03.685556 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134d92fd-7766-457e-bf67-cd133c24ff6f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:07:03.696725 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134d92fd-7766-457e-bf67-cd133c24ff6f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:07:03.698252 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134d92fd-7766-457e-bf67-cd133c24ff6f service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:07:03.710101 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 758/1533] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:07:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 15:07:07.744662 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2423a416-92a3-45eb-b72b-85ecdde515be None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:07:07.745326 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2423a416-92a3-45eb-b72b-85ecdde515be None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:07:07.745326 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2423a416-92a3-45eb-b72b-85ecdde515be None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:07:07.746642 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 776/1534] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 26 15:07:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 15:07:07.753013 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee84b069-5efc-4088-bd9d-272b5003f130 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:07:07.753564 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee84b069-5efc-4088-bd9d-272b5003f130 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:07:07.754229 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee84b069-5efc-4088-bd9d-272b5003f130 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:07:07.776114 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-ee84b069-5efc-4088-bd9d-272b5003f130 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:07:07.777517 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-ee84b069-5efc-4088-bd9d-272b5003f130 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:07:07.787777 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 759/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 26 15:07:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:07:08.067112 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2423a416-92a3-45eb-b72b-85ecdde515be None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:07:08.069660 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-2423a416-92a3-45eb-b72b-85ecdde515be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:07:08.151047 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2423a416-92a3-45eb-b72b-85ecdde515be None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:07:08.156079 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b5f2c0-c080-4541-a9b0-6f4da958f092 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5b5b35614a547d783e6e77ddb8c1c03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:07:08.156735 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b5f2c0-c080-4541-a9b0-6f4da958f092 service cinder] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:07:08.157015 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b5f2c0-c080-4541-a9b0-6f4da958f092 service cinder] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:07:08.157290 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b5f2c0-c080-4541-a9b0-6f4da958f092 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:07:08.169222 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b5f2c0-c080-4541-a9b0-6f4da958f092 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:07:08.171132 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b5f2c0-c080-4541-a9b0-6f4da958f092 service cinder] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:07:08.182747 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 777/1536] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:07:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 15:07:10.549658 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a74902-c0aa-4b88-8ef6-749af530ddf4 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:07:10.550721 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a74902-c0aa-4b88-8ef6-749af530ddf4 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:07:10.550847 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a74902-c0aa-4b88-8ef6-749af530ddf4 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:07:10.552184 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 760/1537] 127.0.0.1 () {54 vars in 916 bytes} [Thu May 26 15:07:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 15:07:10.558193 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2026dc-0e5e-4f3d-9b62-33b3a21ab32c None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:07:10.558685 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2026dc-0e5e-4f3d-9b62-33b3a21ab32c None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:07:10.558813 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2026dc-0e5e-4f3d-9b62-33b3a21ab32c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:07:10.579526 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-0a2026dc-0e5e-4f3d-9b62-33b3a21ab32c None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:07:10.580940 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-0a2026dc-0e5e-4f3d-9b62-33b3a21ab32c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:07:10.591678 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 778/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Thu May 26 15:07:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:07:10.875037 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a74902-c0aa-4b88-8ef6-749af530ddf4 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:07:10.877143 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-65a74902-c0aa-4b88-8ef6-749af530ddf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:07:10.885122 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=61 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:07:10.893546 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a74902-c0aa-4b88-8ef6-749af530ddf4 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:07:10.898195 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01363f54-80b8-4194-a890-6f0aa2a71194 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5b5b35614a547d783e6e77ddb8c1c03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:07:10.899056 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01363f54-80b8-4194-a890-6f0aa2a71194 service cinder] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:07:10.899351 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01363f54-80b8-4194-a890-6f0aa2a71194 service cinder] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:07:10.899729 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01363f54-80b8-4194-a890-6f0aa2a71194 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:07:10.913021 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01363f54-80b8-4194-a890-6f0aa2a71194 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:07:10.914953 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01363f54-80b8-4194-a890-6f0aa2a71194 service cinder] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:07:10.927246 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 761/1539] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:07:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 15:07:20.573154 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=36 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:07:20.921669 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=5 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:08:45.307275 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fa545a-6b8c-4385-a87a-ad1d46fda012 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:08:45.307915 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fa545a-6b8c-4385-a87a-ad1d46fda012 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:08:45.307915 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fa545a-6b8c-4385-a87a-ad1d46fda012 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:08:45.355033 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-66fa545a-6b8c-4385-a87a-ad1d46fda012 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:08:45.356312 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-66fa545a-6b8c-4385-a87a-ad1d46fda012 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:08:45.371312 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 779/1540] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:08:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:08:45.388998 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58e5f92-338c-4ee2-8110-f014c0eaa7c5 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:08:45.389689 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58e5f92-338c-4ee2-8110-f014c0eaa7c5 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:08:45.389689 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58e5f92-338c-4ee2-8110-f014c0eaa7c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:08:45.449357 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-b58e5f92-338c-4ee2-8110-f014c0eaa7c5 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:08:45.450641 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-b58e5f92-338c-4ee2-8110-f014c0eaa7c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:08:45.472783 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 762/1541] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:08:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 15:08:45.486179 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fa545a-6b8c-4385-a87a-ad1d46fda012 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:08:45.504863 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fa545a-6b8c-4385-a87a-ad1d46fda012 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:08:45.515983 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a795b5b-8fd0-40bf-a607-2962456903cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:08:45.516861 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a795b5b-8fd0-40bf-a607-2962456903cc admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:08:45.517151 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a795b5b-8fd0-40bf-a607-2962456903cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:08:45.517430 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a795b5b-8fd0-40bf-a607-2962456903cc admin admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:08:45.518292 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a795b5b-8fd0-40bf-a607-2962456903cc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 15:08:45.518610 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a795b5b-8fd0-40bf-a607-2962456903cc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:08:45.520462 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: WARNING py.warnings [None req-3a795b5b-8fd0-40bf-a607-2962456903cc admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 26 15:08:45.520462 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: warnings.warn(msg) May 26 15:08:45.520462 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: May 26 15:08:45.525996 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a795b5b-8fd0-40bf-a607-2962456903cc admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:08:45.550794 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 780/1542] 10.222.0.46 () {58 vars in 1152 bytes} [Thu May 26 15:08:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 15:08:45.579918 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58e5f92-338c-4ee2-8110-f014c0eaa7c5 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:08:45.597095 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58e5f92-338c-4ee2-8110-f014c0eaa7c5 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:08:45.615879 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9785673-0464-44e5-9dff-3cad38dac084 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:08:45.616490 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9785673-0464-44e5-9dff-3cad38dac084 admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:08:45.616780 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9785673-0464-44e5-9dff-3cad38dac084 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:08:45.617064 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9785673-0464-44e5-9dff-3cad38dac084 admin admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:08:45.621484 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9785673-0464-44e5-9dff-3cad38dac084 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:08:45.621484 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: WARNING py.warnings [None req-a9785673-0464-44e5-9dff-3cad38dac084 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 26 15:08:45.621484 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: warnings.warn(msg) May 26 15:08:45.621484 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: May 26 15:08:45.627133 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9785673-0464-44e5-9dff-3cad38dac084 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:08:45.707580 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987098ec-8218-4ffc-bdac-e5bf1cf2c288 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:08:45.707580 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987098ec-8218-4ffc-bdac-e5bf1cf2c288 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:08:45.707934 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987098ec-8218-4ffc-bdac-e5bf1cf2c288 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:08:45.723890 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 763/1543] 10.222.0.46 () {60 vars in 1152 bytes} [Thu May 26 15:08:45 2022] POST /identity/v3/projects => generated 383 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 15:08:45.731394 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9785673-0464-44e5-9dff-3cad38dac084 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:08:45.753920 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-987098ec-8218-4ffc-bdac-e5bf1cf2c288 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:08:45.756062 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-987098ec-8218-4ffc-bdac-e5bf1cf2c288 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:08:45.835875 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9785673-0464-44e5-9dff-3cad38dac084 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:08:45.844462 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a75127-60e6-4cfc-a7db-8da297ab9e9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:08:45.846174 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a75127-60e6-4cfc-a7db-8da297ab9e9e admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:08:45.846174 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a75127-60e6-4cfc-a7db-8da297ab9e9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:08:45.846174 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a75127-60e6-4cfc-a7db-8da297ab9e9e admin admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:08:45.846779 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a75127-60e6-4cfc-a7db-8da297ab9e9e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:08:45.849302 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57a75127-60e6-4cfc-a7db-8da297ab9e9e admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:08:45.877502 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 764/1544] 10.222.0.46 () {60 vars in 1143 bytes} [Thu May 26 15:08:45 2022] POST /identity/v3/users => generated 292 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 15:08:45.878803 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602ae4b0-d61a-42e2-ba84-464a59a0dd8c None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:08:45.879199 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602ae4b0-d61a-42e2-ba84-464a59a0dd8c None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:08:45.879548 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602ae4b0-d61a-42e2-ba84-464a59a0dd8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:08:45.883022 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 781/1545] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:08:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:08:45.885224 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781fcf8e-c162-49d5-b316-238d4ef3af80 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:08:45.885551 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781fcf8e-c162-49d5-b316-238d4ef3af80 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:08:45.885839 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781fcf8e-c162-49d5-b316-238d4ef3af80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:08:45.903372 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-602ae4b0-d61a-42e2-ba84-464a59a0dd8c None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:08:45.905069 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-602ae4b0-d61a-42e2-ba84-464a59a0dd8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:08:45.915673 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 765/1546] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:08:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:08:45.917567 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd74f5d-503e-4308-a255-33a18fb343e7 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:08:45.918128 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd74f5d-503e-4308-a255-33a18fb343e7 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:08:45.918660 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd74f5d-503e-4308-a255-33a18fb343e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:08:45.921629 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-781fcf8e-c162-49d5-b316-238d4ef3af80 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:08:45.923657 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-781fcf8e-c162-49d5-b316-238d4ef3af80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:08:45.934856 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 782/1547] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:08:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:08:45.936336 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd700a8-da2f-446b-b8a9-6c95eb264b95 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:08:45.937249 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd700a8-da2f-446b-b8a9-6c95eb264b95 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:08:45.937604 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd700a8-da2f-446b-b8a9-6c95eb264b95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:08:45.954112 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-1bd74f5d-503e-4308-a255-33a18fb343e7 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:08:45.955989 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-1bd74f5d-503e-4308-a255-33a18fb343e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:08:45.959590 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-0bd700a8-da2f-446b-b8a9-6c95eb264b95 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:08:45.960855 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-0bd700a8-da2f-446b-b8a9-6c95eb264b95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:08:45.967362 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 766/1548] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:08:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:08:45.969152 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f090a86-2209-4548-840b-c6e887bf5f93 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:08:45.969305 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f090a86-2209-4548-840b-c6e887bf5f93 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:08:45.969723 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f090a86-2209-4548-840b-c6e887bf5f93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:08:45.970027 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 783/1549] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:08:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:08:45.970879 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd700a8-da2f-446b-b8a9-6c95eb264b95 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:08:45.991023 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd700a8-da2f-446b-b8a9-6c95eb264b95 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:08:45.995987 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779fe584-69db-4b79-9633-0ba60b7a1f5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:08:45.996376 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-2f090a86-2209-4548-840b-c6e887bf5f93 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:08:45.996674 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779fe584-69db-4b79-9633-0ba60b7a1f5a admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:08:45.996960 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779fe584-69db-4b79-9633-0ba60b7a1f5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:08:45.997237 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779fe584-69db-4b79-9633-0ba60b7a1f5a admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:08:45.997787 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-2f090a86-2209-4548-840b-c6e887bf5f93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:08:45.998012 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779fe584-69db-4b79-9633-0ba60b7a1f5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:08:46.000141 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779fe584-69db-4b79-9633-0ba60b7a1f5a admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:08:46.007115 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 767/1550] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:08:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:08:46.009548 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c48044-2b01-47b0-a51f-6dae101bfa7b None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:08:46.009548 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c48044-2b01-47b0-a51f-6dae101bfa7b None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:08:46.009548 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c48044-2b01-47b0-a51f-6dae101bfa7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:08:46.017797 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 784/1551] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 15:08:45 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 15:08:46.017797 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb864073-aa5f-4057-817a-506741d71a7e None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:08:46.017797 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb864073-aa5f-4057-817a-506741d71a7e None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:08:46.017797 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb864073-aa5f-4057-817a-506741d71a7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:08:46.034151 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-39c48044-2b01-47b0-a51f-6dae101bfa7b None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:08:46.035382 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-39c48044-2b01-47b0-a51f-6dae101bfa7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:08:46.040425 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-cb864073-aa5f-4057-817a-506741d71a7e None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:08:46.041678 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-cb864073-aa5f-4057-817a-506741d71a7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:08:46.044706 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 768/1552] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:08:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:08:46.046798 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5b30aa-c74f-42d1-839f-03539db5bf1c None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:08:46.047218 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5b30aa-c74f-42d1-839f-03539db5bf1c None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:08:46.047649 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5b30aa-c74f-42d1-839f-03539db5bf1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:08:46.055845 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 785/1553] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:08:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:08:46.055845 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def1adc7-5bd7-42a3-8bf5-9125f7818c66 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:08:46.055845 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def1adc7-5bd7-42a3-8bf5-9125f7818c66 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:08:46.055845 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def1adc7-5bd7-42a3-8bf5-9125f7818c66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:08:46.073076 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-def1adc7-5bd7-42a3-8bf5-9125f7818c66 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:08:46.073841 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-0c5b30aa-c74f-42d1-839f-03539db5bf1c None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:08:46.074332 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-def1adc7-5bd7-42a3-8bf5-9125f7818c66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:08:46.075087 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-0c5b30aa-c74f-42d1-839f-03539db5bf1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:08:46.083134 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 786/1554] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:08:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:08:46.084016 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 769/1555] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:08:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:08:46.085402 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5923ff08-40ac-4a46-b9d7-d4c79ed3486d None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:08:46.085865 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5923ff08-40ac-4a46-b9d7-d4c79ed3486d None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:08:46.086224 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5923ff08-40ac-4a46-b9d7-d4c79ed3486d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:08:46.086937 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cfcf87-2bba-4716-8212-5f180f54156b None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:08:46.087969 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cfcf87-2bba-4716-8212-5f180f54156b None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:08:46.088258 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cfcf87-2bba-4716-8212-5f180f54156b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:08:46.106588 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-36cfcf87-2bba-4716-8212-5f180f54156b None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:08:46.107818 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-36cfcf87-2bba-4716-8212-5f180f54156b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:08:46.107944 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-5923ff08-40ac-4a46-b9d7-d4c79ed3486d None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:08:46.109224 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-5923ff08-40ac-4a46-b9d7-d4c79ed3486d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:08:46.116607 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 787/1556] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:08:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:08:46.117733 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 770/1557] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:08:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:08:46.118580 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5923ff08-40ac-4a46-b9d7-d4c79ed3486d None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:08:46.121615 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62f39ed-d155-4b1a-a7c0-a469216baad6 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:08:46.121916 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62f39ed-d155-4b1a-a7c0-a469216baad6 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:08:46.122205 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62f39ed-d155-4b1a-a7c0-a469216baad6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:08:46.133023 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5923ff08-40ac-4a46-b9d7-d4c79ed3486d None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:08:46.138090 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4ce1bd-f06a-4f7d-a7df-799c41a776c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:08:46.138814 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4ce1bd-f06a-4f7d-a7df-799c41a776c0 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:08:46.139104 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4ce1bd-f06a-4f7d-a7df-799c41a776c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:08:46.139389 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4ce1bd-f06a-4f7d-a7df-799c41a776c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:08:46.140815 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4ce1bd-f06a-4f7d-a7df-799c41a776c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:08:46.142531 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4ce1bd-f06a-4f7d-a7df-799c41a776c0 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:08:46.143294 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-f62f39ed-d155-4b1a-a7c0-a469216baad6 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:08:46.144904 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-f62f39ed-d155-4b1a-a7c0-a469216baad6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:08:46.158274 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 771/1558] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 15:08:46 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 15:08:46.158274 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bb5268-64e8-4b5a-9561-da9b4e5dd0f5 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:08:46.158274 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bb5268-64e8-4b5a-9561-da9b4e5dd0f5 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:08:46.158878 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 788/1559] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:08:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:08:46.159034 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bb5268-64e8-4b5a-9561-da9b4e5dd0f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:08:46.159176 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62f39ed-d155-4b1a-a7c0-a469216baad6 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:08:46.170930 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62f39ed-d155-4b1a-a7c0-a469216baad6 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:08:46.175750 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8db9404-13d5-48c3-8595-12c0e55125e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:08:46.176460 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8db9404-13d5-48c3-8595-12c0e55125e7 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:08:46.176746 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8db9404-13d5-48c3-8595-12c0e55125e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:08:46.177025 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8db9404-13d5-48c3-8595-12c0e55125e7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:08:46.178995 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8db9404-13d5-48c3-8595-12c0e55125e7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 15:08:46.179176 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-50bb5268-64e8-4b5a-9561-da9b4e5dd0f5 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:08:46.179290 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8db9404-13d5-48c3-8595-12c0e55125e7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:08:46.181822 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-50bb5268-64e8-4b5a-9561-da9b4e5dd0f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:08:46.181992 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8db9404-13d5-48c3-8595-12c0e55125e7 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:08:46.189532 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 772/1560] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:08:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:08:46.190469 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bb5268-64e8-4b5a-9561-da9b4e5dd0f5 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:08:46.201268 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bb5268-64e8-4b5a-9561-da9b4e5dd0f5 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:08:46.204136 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 789/1561] 10.222.0.46 () {58 vars in 1152 bytes} [Thu May 26 15:08:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 15:08:46.205095 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8db9404-13d5-48c3-8595-12c0e55125e7 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:08:46.206961 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f29078a-31c3-490c-82aa-301374a052f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:08:46.207670 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f29078a-31c3-490c-82aa-301374a052f4 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:08:46.207954 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f29078a-31c3-490c-82aa-301374a052f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:08:46.208245 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f29078a-31c3-490c-82aa-301374a052f4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:08:46.209075 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f29078a-31c3-490c-82aa-301374a052f4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 15:08:46.209390 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f29078a-31c3-490c-82aa-301374a052f4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:08:46.211644 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f29078a-31c3-490c-82aa-301374a052f4 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:08:46.215991 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8db9404-13d5-48c3-8595-12c0e55125e7 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:08:46.220896 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f98e5bc-64b5-46fc-80ca-7509aa97236f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:08:46.221612 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f98e5bc-64b5-46fc-80ca-7509aa97236f admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:08:46.221903 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f98e5bc-64b5-46fc-80ca-7509aa97236f admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:08:46.222177 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f98e5bc-64b5-46fc-80ca-7509aa97236f admin admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:08:46.222906 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f98e5bc-64b5-46fc-80ca-7509aa97236f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 15:08:46.223222 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f98e5bc-64b5-46fc-80ca-7509aa97236f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:08:46.224898 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 773/1562] 10.222.0.46 () {58 vars in 1152 bytes} [Thu May 26 15:08:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 15:08:46.225244 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f98e5bc-64b5-46fc-80ca-7509aa97236f admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:08:46.225999 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f29078a-31c3-490c-82aa-301374a052f4 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:08:46.241867 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f29078a-31c3-490c-82aa-301374a052f4 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:08:46.245474 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 790/1563] 10.222.0.46 () {58 vars in 1152 bytes} [Thu May 26 15:08:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 15:08:46.247013 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f98e5bc-64b5-46fc-80ca-7509aa97236f admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:08:46.250958 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bbb998-544f-4bc0-8e2a-8b3c00f0f6b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:08:46.252372 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bbb998-544f-4bc0-8e2a-8b3c00f0f6b7 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:08:46.253172 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bbb998-544f-4bc0-8e2a-8b3c00f0f6b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:08:46.253915 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bbb998-544f-4bc0-8e2a-8b3c00f0f6b7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:08:46.255212 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bbb998-544f-4bc0-8e2a-8b3c00f0f6b7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 15:08:46.255720 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bbb998-544f-4bc0-8e2a-8b3c00f0f6b7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:08:46.258272 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bbb998-544f-4bc0-8e2a-8b3c00f0f6b7 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:08:46.264234 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f98e5bc-64b5-46fc-80ca-7509aa97236f admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:08:46.271436 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2b2913-6584-4a29-aa43-bb8c5c2df690 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:08:46.272537 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2b2913-6584-4a29-aa43-bb8c5c2df690 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:08:46.273008 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2b2913-6584-4a29-aa43-bb8c5c2df690 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:08:46.273501 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2b2913-6584-4a29-aa43-bb8c5c2df690 admin admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:08:46.274587 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2b2913-6584-4a29-aa43-bb8c5c2df690 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 15:08:46.275107 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2b2913-6584-4a29-aa43-bb8c5c2df690 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:08:46.277895 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff2b2913-6584-4a29-aa43-bb8c5c2df690 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:08:46.286627 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 774/1564] 10.222.0.46 () {58 vars in 1152 bytes} [Thu May 26 15:08:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 15:08:46.286627 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bbb998-544f-4bc0-8e2a-8b3c00f0f6b7 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:08:46.293040 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 791/1565] 10.222.0.46 () {58 vars in 1152 bytes} [Thu May 26 15:08:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 15:08:46.293959 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2b2913-6584-4a29-aa43-bb8c5c2df690 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:08:46.306463 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff2b2913-6584-4a29-aa43-bb8c5c2df690 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:08:46.307160 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bbb998-544f-4bc0-8e2a-8b3c00f0f6b7 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:08:46.311528 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b45fb9-6d56-46c6-a027-cd8fc38031af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:08:46.312716 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b45fb9-6d56-46c6-a027-cd8fc38031af admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:08:46.313004 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b45fb9-6d56-46c6-a027-cd8fc38031af admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:08:46.313284 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b45fb9-6d56-46c6-a027-cd8fc38031af admin admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:08:46.313526 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25ea09c-2c28-4abe-80b2-7820f87db527 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:08:46.314061 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b45fb9-6d56-46c6-a027-cd8fc38031af admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 15:08:46.314368 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b45fb9-6d56-46c6-a027-cd8fc38031af admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:08:46.314472 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25ea09c-2c28-4abe-80b2-7820f87db527 admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:08:46.315734 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25ea09c-2c28-4abe-80b2-7820f87db527 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:08:46.315734 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25ea09c-2c28-4abe-80b2-7820f87db527 admin admin] PATH_INFO: `/v3/projects/595a3d59c45f452d935dcec7a8938828/users/eb0888d2b5024d558d15fe47e2d34ccb/roles/d4088d1845a24fbd9bd8975ce0b1d3d0` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:08:46.315998 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b45fb9-6d56-46c6-a027-cd8fc38031af admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:08:46.330722 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 792/1566] 10.222.0.46 () {58 vars in 1152 bytes} [Thu May 26 15:08:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 15:08:46.331681 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b45fb9-6d56-46c6-a027-cd8fc38031af admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:08:46.335690 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25ea09c-2c28-4abe-80b2-7820f87db527 admin admin] RBAC: Authorizing `identity:create_grant(project_id=595a3d59c45f452d935dcec7a8938828, user_id=eb0888d2b5024d558d15fe47e2d34ccb, role_id=d4088d1845a24fbd9bd8975ce0b1d3d0)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:08:46.338340 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: WARNING py.warnings [None req-c25ea09c-2c28-4abe-80b2-7820f87db527 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 26 15:08:46.338340 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: warnings.warn(msg) May 26 15:08:46.338340 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: May 26 15:08:46.341723 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25ea09c-2c28-4abe-80b2-7820f87db527 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:08:46.347415 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b45fb9-6d56-46c6-a027-cd8fc38031af admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:08:46.353968 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f078ce5e-4ac6-470f-80cc-d3a0a2678486 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:08:46.354818 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f078ce5e-4ac6-470f-80cc-d3a0a2678486 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:08:46.355237 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f078ce5e-4ac6-470f-80cc-d3a0a2678486 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:08:46.355766 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f078ce5e-4ac6-470f-80cc-d3a0a2678486 admin admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:08:46.356673 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f078ce5e-4ac6-470f-80cc-d3a0a2678486 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 15:08:46.357130 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f078ce5e-4ac6-470f-80cc-d3a0a2678486 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:08:46.359870 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f078ce5e-4ac6-470f-80cc-d3a0a2678486 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:08:46.362815 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 775/1567] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 15:08:46 2022] PUT /identity/v3/projects/595a3d59c45f452d935dcec7a8938828/users/eb0888d2b5024d558d15fe47e2d34ccb/roles/d4088d1845a24fbd9bd8975ce0b1d3d0 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:08:46.363946 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25ea09c-2c28-4abe-80b2-7820f87db527 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:08:46.380007 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:08:46.386340 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 793/1568] 10.222.0.46 () {58 vars in 1152 bytes} [Thu May 26 15:08:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 15:08:46.387440 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f078ce5e-4ac6-470f-80cc-d3a0a2678486 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:08:46.401037 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f078ce5e-4ac6-470f-80cc-d3a0a2678486 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:08:46.426830 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:08:46.446490 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25ea09c-2c28-4abe-80b2-7820f87db527 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:08:46.452609 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d670f4-cf55-40ab-af42-4fcaf654891a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:08:46.453311 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d670f4-cf55-40ab-af42-4fcaf654891a admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:08:46.453637 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d670f4-cf55-40ab-af42-4fcaf654891a admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:08:46.453926 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d670f4-cf55-40ab-af42-4fcaf654891a admin admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:08:46.454615 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d670f4-cf55-40ab-af42-4fcaf654891a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:08:46.456546 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d670f4-cf55-40ab-af42-4fcaf654891a admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:08:46.476399 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e21979-1a28-47e2-92ea-795c005e89eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:08:46.477079 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e21979-1a28-47e2-92ea-795c005e89eb admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:08:46.477403 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e21979-1a28-47e2-92ea-795c005e89eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:08:46.477717 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e21979-1a28-47e2-92ea-795c005e89eb admin admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:08:46.478425 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e21979-1a28-47e2-92ea-795c005e89eb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:08:46.480439 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e21979-1a28-47e2-92ea-795c005e89eb admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:08:46.486719 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 776/1569] 10.222.0.46 () {60 vars in 1152 bytes} [Thu May 26 15:08:46 2022] POST /identity/v3/projects => generated 375 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 15:08:46.487692 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d670f4-cf55-40ab-af42-4fcaf654891a admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:08:46.517577 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 794/1570] 10.222.0.46 () {60 vars in 1152 bytes} [Thu May 26 15:08:46 2022] POST /identity/v3/projects => generated 403 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 15:08:46.517577 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e21979-1a28-47e2-92ea-795c005e89eb admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:08:46.564402 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d670f4-cf55-40ab-af42-4fcaf654891a admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:08:46.590146 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e21979-1a28-47e2-92ea-795c005e89eb admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:08:46.595700 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aba0791-4d0c-4d31-a096-610e7205d31a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:08:46.596481 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aba0791-4d0c-4d31-a096-610e7205d31a admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:08:46.596901 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aba0791-4d0c-4d31-a096-610e7205d31a admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:08:46.597248 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aba0791-4d0c-4d31-a096-610e7205d31a admin admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:08:46.598149 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aba0791-4d0c-4d31-a096-610e7205d31a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:08:46.600214 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aba0791-4d0c-4d31-a096-610e7205d31a admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:08:46.634944 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 795/1571] 10.222.0.46 () {60 vars in 1152 bytes} [Thu May 26 15:08:46 2022] POST /identity/v3/projects => generated 403 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 15:08:46.638661 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aba0791-4d0c-4d31-a096-610e7205d31a admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:08:46.649188 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7a3532-9e3f-4e8c-9f8c-bb59e91f1e4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:08:46.649927 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7a3532-9e3f-4e8c-9f8c-bb59e91f1e4e admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:08:46.650246 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7a3532-9e3f-4e8c-9f8c-bb59e91f1e4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:08:46.650536 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7a3532-9e3f-4e8c-9f8c-bb59e91f1e4e admin admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:08:46.651237 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7a3532-9e3f-4e8c-9f8c-bb59e91f1e4e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:08:46.653290 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7a3532-9e3f-4e8c-9f8c-bb59e91f1e4e admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:08:46.688775 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 777/1572] 10.222.0.46 () {60 vars in 1152 bytes} [Thu May 26 15:08:46 2022] POST /identity/v3/projects => generated 389 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 15:08:46.690015 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7a3532-9e3f-4e8c-9f8c-bb59e91f1e4e admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:08:46.720747 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aba0791-4d0c-4d31-a096-610e7205d31a admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:08:46.730601 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:08:46.771388 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7a3532-9e3f-4e8c-9f8c-bb59e91f1e4e admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:08:46.776589 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d0086f-21b4-430e-9c36-5ef05b72495b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:08:46.777105 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:08:46.777367 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d0086f-21b4-430e-9c36-5ef05b72495b admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:08:46.777612 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d0086f-21b4-430e-9c36-5ef05b72495b admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:08:46.778027 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d0086f-21b4-430e-9c36-5ef05b72495b admin admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:08:46.778741 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d0086f-21b4-430e-9c36-5ef05b72495b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:08:46.781364 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d0086f-21b4-430e-9c36-5ef05b72495b admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:08:46.801377 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267f4011-db8a-4987-9234-ccf17343e60d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:08:46.802292 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267f4011-db8a-4987-9234-ccf17343e60d admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:08:46.802656 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267f4011-db8a-4987-9234-ccf17343e60d admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:08:46.802994 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267f4011-db8a-4987-9234-ccf17343e60d admin admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:08:46.803829 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267f4011-db8a-4987-9234-ccf17343e60d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:08:46.805931 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267f4011-db8a-4987-9234-ccf17343e60d admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:08:46.822131 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 778/1573] 10.222.0.46 () {60 vars in 1152 bytes} [Thu May 26 15:08:46 2022] POST /identity/v3/projects => generated 383 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 15:08:46.824272 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79834686-c288-48ee-8b59-c60018ca5436 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:08:46.824728 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79834686-c288-48ee-8b59-c60018ca5436 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:08:46.825130 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79834686-c288-48ee-8b59-c60018ca5436 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:08:46.844036 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 796/1574] 10.222.0.46 () {60 vars in 1152 bytes} [Thu May 26 15:08:46 2022] POST /identity/v3/projects => generated 379 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 15:08:46.845309 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267f4011-db8a-4987-9234-ccf17343e60d admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:08:46.862511 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-79834686-c288-48ee-8b59-c60018ca5436 None None] MFA Rules not processed for user `eb0888d2b5024d558d15fe47e2d34ccb`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:08:46.864037 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-79834686-c288-48ee-8b59-c60018ca5436 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:08:46.937293 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267f4011-db8a-4987-9234-ccf17343e60d admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:08:46.944049 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 779/1575] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:08:46 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:08:46.944461 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27be7585-53cf-4c3c-8a95-d3c1746872e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:08:46.945002 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79834686-c288-48ee-8b59-c60018ca5436 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:08:46.945289 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27be7585-53cf-4c3c-8a95-d3c1746872e2 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:08:46.945775 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27be7585-53cf-4c3c-8a95-d3c1746872e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:08:46.946121 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27be7585-53cf-4c3c-8a95-d3c1746872e2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:08:46.947290 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27be7585-53cf-4c3c-8a95-d3c1746872e2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:08:46.951140 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27be7585-53cf-4c3c-8a95-d3c1746872e2 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:08:46.960605 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79834686-c288-48ee-8b59-c60018ca5436 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:08:46.973610 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b06111-4de5-4024-bda9-86245a940364 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:08:46.974692 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b06111-4de5-4024-bda9-86245a940364 admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:08:46.974929 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b06111-4de5-4024-bda9-86245a940364 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:08:46.975280 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b06111-4de5-4024-bda9-86245a940364 admin admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:08:46.976127 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b06111-4de5-4024-bda9-86245a940364 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:08:46.978477 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b06111-4de5-4024-bda9-86245a940364 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:08:47.001616 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 797/1576] 10.222.0.46 () {60 vars in 1152 bytes} [Thu May 26 15:08:46 2022] POST /identity/v3/projects => generated 375 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 15:08:47.003310 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27be7585-53cf-4c3c-8a95-d3c1746872e2 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:08:47.003651 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 780/1577] 10.222.0.46 () {60 vars in 1143 bytes} [Thu May 26 15:08:46 2022] POST /identity/v3/users => generated 288 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 15:08:47.007597 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b06111-4de5-4024-bda9-86245a940364 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:08:47.018335 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b06111-4de5-4024-bda9-86245a940364 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:08:47.088269 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27be7585-53cf-4c3c-8a95-d3c1746872e2 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:08:47.093652 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6a950d-e721-4614-b7cd-f2b52b3af228 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:08:47.094251 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38186cd-1d0f-47a3-a4f7-6e8e328e0966 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:08:47.094403 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6a950d-e721-4614-b7cd-f2b52b3af228 admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:08:47.094615 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6a950d-e721-4614-b7cd-f2b52b3af228 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:08:47.094901 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6a950d-e721-4614-b7cd-f2b52b3af228 admin admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:08:47.095038 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38186cd-1d0f-47a3-a4f7-6e8e328e0966 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:08:47.095276 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38186cd-1d0f-47a3-a4f7-6e8e328e0966 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:08:47.095594 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6a950d-e721-4614-b7cd-f2b52b3af228 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:08:47.095695 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38186cd-1d0f-47a3-a4f7-6e8e328e0966 admin admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:08:47.096434 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38186cd-1d0f-47a3-a4f7-6e8e328e0966 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:08:47.097600 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: WARNING py.warnings [None req-b38186cd-1d0f-47a3-a4f7-6e8e328e0966 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 26 15:08:47.097600 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: warnings.warn(msg) May 26 15:08:47.097600 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: May 26 15:08:47.097901 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab6a950d-e721-4614-b7cd-f2b52b3af228 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:08:47.099149 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38186cd-1d0f-47a3-a4f7-6e8e328e0966 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:08:47.123748 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 798/1578] 10.222.0.46 () {60 vars in 1143 bytes} [Thu May 26 15:08:47 2022] POST /identity/v3/users => generated 302 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 15:08:47.123958 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 781/1579] 10.222.0.46 () {60 vars in 1143 bytes} [Thu May 26 15:08:47 2022] POST /identity/v3/users => generated 302 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 15:08:47.124390 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38186cd-1d0f-47a3-a4f7-6e8e328e0966 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:08:47.124512 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6a950d-e721-4614-b7cd-f2b52b3af228 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:08:47.135174 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38186cd-1d0f-47a3-a4f7-6e8e328e0966 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:08:47.136064 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6a950d-e721-4614-b7cd-f2b52b3af228 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:08:47.140694 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ca929e-3d00-4558-835b-cd1f058c0706 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:08:47.141410 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ca929e-3d00-4558-835b-cd1f058c0706 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:08:47.142684 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ca929e-3d00-4558-835b-cd1f058c0706 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:08:47.142684 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ca929e-3d00-4558-835b-cd1f058c0706 admin admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:08:47.142910 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd287dc-380b-48b8-bee8-bbf550450c14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:08:47.145873 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ca929e-3d00-4558-835b-cd1f058c0706 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:08:47.146002 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd287dc-380b-48b8-bee8-bbf550450c14 admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:08:47.146002 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd287dc-380b-48b8-bee8-bbf550450c14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:08:47.146002 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd287dc-380b-48b8-bee8-bbf550450c14 admin admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:08:47.146002 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd287dc-380b-48b8-bee8-bbf550450c14 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:08:47.146296 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ca929e-3d00-4558-835b-cd1f058c0706 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:08:47.147217 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bd287dc-380b-48b8-bee8-bbf550450c14 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:08:47.167842 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=90,keystone:INSERT=10 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:08:47.175919 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 799/1580] 10.222.0.46 () {60 vars in 1143 bytes} [Thu May 26 15:08:47 2022] POST /identity/v3/users => generated 295 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 15:08:47.175919 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ca929e-3d00-4558-835b-cd1f058c0706 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:08:47.176198 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 782/1581] 10.222.0.46 () {60 vars in 1143 bytes} [Thu May 26 15:08:47 2022] POST /identity/v3/users => generated 292 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 15:08:47.179188 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c2cb43-6685-405c-91a0-26aebcc41c59 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:08:47.180102 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c2cb43-6685-405c-91a0-26aebcc41c59 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:08:47.180377 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c2cb43-6685-405c-91a0-26aebcc41c59 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:08:47.183696 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 783/1582] 10.222.0.46 () {54 vars in 961 bytes} [Thu May 26 15:08:47 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 15:08:47.184097 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c2cb43-6685-405c-91a0-26aebcc41c59 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:08:47.188238 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ca929e-3d00-4558-835b-cd1f058c0706 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:08:47.193838 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c582571-4f75-41a4-a040-4fb688f62003 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:08:47.194678 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c582571-4f75-41a4-a040-4fb688f62003 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:08:47.195036 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c582571-4f75-41a4-a040-4fb688f62003 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:08:47.195347 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c582571-4f75-41a4-a040-4fb688f62003 admin admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:08:47.196185 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c582571-4f75-41a4-a040-4fb688f62003 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:08:47.198360 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c582571-4f75-41a4-a040-4fb688f62003 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:08:47.202800 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c2cb43-6685-405c-91a0-26aebcc41c59 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:08:47.207794 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7b3106-f38a-40f8-bcef-6a1b44a1448f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:08:47.209256 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7b3106-f38a-40f8-bcef-6a1b44a1448f admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:08:47.209256 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7b3106-f38a-40f8-bcef-6a1b44a1448f admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:08:47.209256 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7b3106-f38a-40f8-bcef-6a1b44a1448f admin admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:08:47.209923 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7b3106-f38a-40f8-bcef-6a1b44a1448f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:08:47.211900 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7b3106-f38a-40f8-bcef-6a1b44a1448f admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:08:47.224498 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 800/1583] 10.222.0.46 () {60 vars in 1143 bytes} [Thu May 26 15:08:47 2022] POST /identity/v3/users => generated 290 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 15:08:47.225355 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c582571-4f75-41a4-a040-4fb688f62003 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:08:47.237426 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c582571-4f75-41a4-a040-4fb688f62003 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:08:47.243392 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 784/1584] 10.222.0.46 () {60 vars in 1143 bytes} [Thu May 26 15:08:47 2022] POST /identity/v3/users => generated 288 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 15:08:47.243668 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cee16a-ef29-4c4a-b574-0c5eda716645 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:08:47.244217 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7b3106-f38a-40f8-bcef-6a1b44a1448f admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:08:47.244360 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cee16a-ef29-4c4a-b574-0c5eda716645 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:08:47.244839 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cee16a-ef29-4c4a-b574-0c5eda716645 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:08:47.245445 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cee16a-ef29-4c4a-b574-0c5eda716645 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:08:47.246725 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5cee16a-ef29-4c4a-b574-0c5eda716645 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:08:47.249653 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5cee16a-ef29-4c4a-b574-0c5eda716645 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:08:47.251472 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=89,keystone:INSERT=11 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:08:47.257018 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7b3106-f38a-40f8-bcef-6a1b44a1448f admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:08:47.264206 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb48f0d-0586-4868-b2c8-96e3057ff0f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:08:47.266091 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb48f0d-0586-4868-b2c8-96e3057ff0f1 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:08:47.266560 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb48f0d-0586-4868-b2c8-96e3057ff0f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:08:47.266972 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb48f0d-0586-4868-b2c8-96e3057ff0f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:08:47.268032 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb48f0d-0586-4868-b2c8-96e3057ff0f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:08:47.268575 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 801/1585] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 15:08:47 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 15:08:47.269578 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cee16a-ef29-4c4a-b574-0c5eda716645 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:08:47.271112 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb48f0d-0586-4868-b2c8-96e3057ff0f1 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:08:47.280788 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cee16a-ef29-4c4a-b574-0c5eda716645 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:08:47.285799 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f80f754-ca99-49dc-9a92-c6d387236c0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:08:47.286078 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 785/1586] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 15:08:47 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 15:08:47.286563 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f80f754-ca99-49dc-9a92-c6d387236c0c admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:08:47.286849 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f80f754-ca99-49dc-9a92-c6d387236c0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:08:47.287010 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb48f0d-0586-4868-b2c8-96e3057ff0f1 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:08:47.287170 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f80f754-ca99-49dc-9a92-c6d387236c0c admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:08:47.288016 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f80f754-ca99-49dc-9a92-c6d387236c0c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:08:47.289789 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f80f754-ca99-49dc-9a92-c6d387236c0c admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:08:47.300071 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb48f0d-0586-4868-b2c8-96e3057ff0f1 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:08:47.301102 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 802/1587] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 15:08:47 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 15:08:47.306320 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f80f754-ca99-49dc-9a92-c6d387236c0c admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:08:47.306501 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36147463-1688-4962-81c0-2f234af6ed08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:08:47.307162 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36147463-1688-4962-81c0-2f234af6ed08 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:08:47.307445 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36147463-1688-4962-81c0-2f234af6ed08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:08:47.307721 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36147463-1688-4962-81c0-2f234af6ed08 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:08:47.308455 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36147463-1688-4962-81c0-2f234af6ed08 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:08:47.310140 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36147463-1688-4962-81c0-2f234af6ed08 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:08:47.316644 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f80f754-ca99-49dc-9a92-c6d387236c0c admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:08:47.321684 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a286478-b65f-4247-9401-9452e75fe211 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:08:47.322403 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a286478-b65f-4247-9401-9452e75fe211 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:08:47.322683 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a286478-b65f-4247-9401-9452e75fe211 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:08:47.322962 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a286478-b65f-4247-9401-9452e75fe211 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:08:47.323287 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 786/1588] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 15:08:47 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 15:08:47.323706 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a286478-b65f-4247-9401-9452e75fe211 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:08:47.325237 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a356b5a2-9756-4c10-a1fe-30c28d2006bc None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:08:47.325426 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a286478-b65f-4247-9401-9452e75fe211 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:08:47.325628 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a356b5a2-9756-4c10-a1fe-30c28d2006bc None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:08:47.325837 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a356b5a2-9756-4c10-a1fe-30c28d2006bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:08:47.340652 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 803/1589] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 15:08:47 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 15:08:47.340652 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a286478-b65f-4247-9401-9452e75fe211 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:08:47.348601 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-a356b5a2-9756-4c10-a1fe-30c28d2006bc None None] MFA Rules not processed for user `d6639353c70449d1b3d10cd6487c3b00`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:08:47.350228 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-a356b5a2-9756-4c10-a1fe-30c28d2006bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:08:47.352913 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a286478-b65f-4247-9401-9452e75fe211 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:08:47.358311 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8c679d-97d6-4caa-bbcc-8d6ddeb3b321 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:08:47.358990 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8c679d-97d6-4caa-bbcc-8d6ddeb3b321 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:08:47.359275 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8c679d-97d6-4caa-bbcc-8d6ddeb3b321 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:08:47.359552 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8c679d-97d6-4caa-bbcc-8d6ddeb3b321 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:08:47.360290 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8c679d-97d6-4caa-bbcc-8d6ddeb3b321 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:08:47.361959 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8c679d-97d6-4caa-bbcc-8d6ddeb3b321 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:08:47.374913 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 804/1590] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 15:08:47 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 15:08:47.375937 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8c679d-97d6-4caa-bbcc-8d6ddeb3b321 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:08:47.386650 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8c679d-97d6-4caa-bbcc-8d6ddeb3b321 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:08:47.392620 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b416ffa5-14b2-46fb-b482-66cc3d1d36af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:08:47.393307 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b416ffa5-14b2-46fb-b482-66cc3d1d36af admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:08:47.393624 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b416ffa5-14b2-46fb-b482-66cc3d1d36af admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:08:47.393912 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b416ffa5-14b2-46fb-b482-66cc3d1d36af admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:08:47.394652 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b416ffa5-14b2-46fb-b482-66cc3d1d36af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:08:47.396367 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b416ffa5-14b2-46fb-b482-66cc3d1d36af admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:08:47.408539 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 805/1591] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 15:08:47 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 15:08:47.409289 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b416ffa5-14b2-46fb-b482-66cc3d1d36af admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:08:47.420727 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b416ffa5-14b2-46fb-b482-66cc3d1d36af admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:08:47.426592 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9d634b-9fd1-4b9b-8254-79aa705f866a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:08:47.427281 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9d634b-9fd1-4b9b-8254-79aa705f866a admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:08:47.427569 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9d634b-9fd1-4b9b-8254-79aa705f866a admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:08:47.427870 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9d634b-9fd1-4b9b-8254-79aa705f866a admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:08:47.428580 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9d634b-9fd1-4b9b-8254-79aa705f866a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:08:47.430271 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9d634b-9fd1-4b9b-8254-79aa705f866a admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:08:47.442788 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 806/1592] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 15:08:47 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 15:08:47.443665 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9d634b-9fd1-4b9b-8254-79aa705f866a admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:08:47.451512 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 787/1593] 10.222.0.46 () {58 vars in 1058 bytes} [Thu May 26 15:08:47 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:08:47.453909 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a356b5a2-9756-4c10-a1fe-30c28d2006bc None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:08:47.459109 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9d634b-9fd1-4b9b-8254-79aa705f866a admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:08:47.467984 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207e8ae5-e89d-4e0a-942d-9a8c13af9fdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:08:47.469066 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207e8ae5-e89d-4e0a-942d-9a8c13af9fdf admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:08:47.469588 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207e8ae5-e89d-4e0a-942d-9a8c13af9fdf admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:08:47.470071 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207e8ae5-e89d-4e0a-942d-9a8c13af9fdf admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:08:47.471272 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207e8ae5-e89d-4e0a-942d-9a8c13af9fdf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:08:47.472801 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a356b5a2-9756-4c10-a1fe-30c28d2006bc None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:08:47.474431 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207e8ae5-e89d-4e0a-942d-9a8c13af9fdf admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:08:47.479798 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ad4d67-88e3-40ee-ab34-ca4554f57b9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:08:47.480873 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ad4d67-88e3-40ee-ab34-ca4554f57b9f admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:08:47.481338 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ad4d67-88e3-40ee-ab34-ca4554f57b9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:08:47.481830 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ad4d67-88e3-40ee-ab34-ca4554f57b9f admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:08:47.483097 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ad4d67-88e3-40ee-ab34-ca4554f57b9f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:08:47.485959 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ad4d67-88e3-40ee-ab34-ca4554f57b9f admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:08:47.498215 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 807/1594] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 15:08:47 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 15:08:47.498215 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207e8ae5-e89d-4e0a-942d-9a8c13af9fdf admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:08:47.506568 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 788/1595] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 15:08:47 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 15:08:47.506568 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ad4d67-88e3-40ee-ab34-ca4554f57b9f admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:08:47.510383 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207e8ae5-e89d-4e0a-942d-9a8c13af9fdf admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:08:47.514340 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ad4d67-88e3-40ee-ab34-ca4554f57b9f admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:08:47.516756 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfba57eb-d895-40fd-9ba5-2ae96a15307e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:08:47.517964 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfba57eb-d895-40fd-9ba5-2ae96a15307e admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:08:47.518474 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfba57eb-d895-40fd-9ba5-2ae96a15307e admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:08:47.518953 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfba57eb-d895-40fd-9ba5-2ae96a15307e admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:08:47.519998 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b7e09b-5e2a-44a8-8976-355dded4c38b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:08:47.520230 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfba57eb-d895-40fd-9ba5-2ae96a15307e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:08:47.520677 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b7e09b-5e2a-44a8-8976-355dded4c38b admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:08:47.521043 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b7e09b-5e2a-44a8-8976-355dded4c38b admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:08:47.521329 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b7e09b-5e2a-44a8-8976-355dded4c38b admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:08:47.522105 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b7e09b-5e2a-44a8-8976-355dded4c38b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:08:47.523153 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfba57eb-d895-40fd-9ba5-2ae96a15307e admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:08:47.523718 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b7e09b-5e2a-44a8-8976-355dded4c38b admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:08:47.538941 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 789/1596] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 15:08:47 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 15:08:47.539824 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b7e09b-5e2a-44a8-8976-355dded4c38b admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:08:47.544395 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 808/1597] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 15:08:47 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 15:08:47.544395 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfba57eb-d895-40fd-9ba5-2ae96a15307e admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:08:47.551702 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b7e09b-5e2a-44a8-8976-355dded4c38b admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:08:47.553912 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfba57eb-d895-40fd-9ba5-2ae96a15307e admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:08:47.556728 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417b33de-9972-4210-a71a-300cd3e51385 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:08:47.557523 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417b33de-9972-4210-a71a-300cd3e51385 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:08:47.557817 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417b33de-9972-4210-a71a-300cd3e51385 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:08:47.558101 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417b33de-9972-4210-a71a-300cd3e51385 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:08:47.558712 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd06397-885f-45f5-aec7-bf443b219829 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:08:47.558935 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417b33de-9972-4210-a71a-300cd3e51385 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:08:47.559460 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd06397-885f-45f5-aec7-bf443b219829 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:08:47.559747 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd06397-885f-45f5-aec7-bf443b219829 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:08:47.560029 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd06397-885f-45f5-aec7-bf443b219829 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:08:47.560753 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417b33de-9972-4210-a71a-300cd3e51385 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:08:47.560926 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd06397-885f-45f5-aec7-bf443b219829 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:08:47.563494 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd06397-885f-45f5-aec7-bf443b219829 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:08:47.580067 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 790/1598] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 15:08:47 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 15:08:47.580310 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 809/1599] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 15:08:47 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 15:08:47.580413 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417b33de-9972-4210-a71a-300cd3e51385 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:08:47.580515 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd06397-885f-45f5-aec7-bf443b219829 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:08:47.591847 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417b33de-9972-4210-a71a-300cd3e51385 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:08:47.593217 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd06397-885f-45f5-aec7-bf443b219829 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:08:47.596789 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16f1607-7958-40e5-9110-1fa75033e0c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:08:47.597720 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16f1607-7958-40e5-9110-1fa75033e0c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:08:47.598070 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16f1607-7958-40e5-9110-1fa75033e0c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:08:47.598358 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16f1607-7958-40e5-9110-1fa75033e0c1 admin admin] PATH_INFO: `/v3/projects/612679bca8894138b317a0c19d8b2069/users/ff29b1cfe4c945a7a7aca87b79fc8075/roles/d4088d1845a24fbd9bd8975ce0b1d3d0` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:08:47.599015 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f16ea1-17c7-42cb-91ca-0394e164fa6c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6639353c70449d1b3d10cd6487c3b00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:08:47.599610 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f16ea1-17c7-42cb-91ca-0394e164fa6c service nova] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:08:47.599894 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f16ea1-17c7-42cb-91ca-0394e164fa6c service nova] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:08:47.600175 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f16ea1-17c7-42cb-91ca-0394e164fa6c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:08:47.614721 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16f1607-7958-40e5-9110-1fa75033e0c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=612679bca8894138b317a0c19d8b2069, user_id=ff29b1cfe4c945a7a7aca87b79fc8075, role_id=d4088d1845a24fbd9bd8975ce0b1d3d0)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:08:47.617409 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16f1607-7958-40e5-9110-1fa75033e0c1 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:08:47.636481 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 791/1600] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 15:08:47 2022] PUT /identity/v3/projects/612679bca8894138b317a0c19d8b2069/users/ff29b1cfe4c945a7a7aca87b79fc8075/roles/d4088d1845a24fbd9bd8975ce0b1d3d0 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:08:47.638837 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16f1607-7958-40e5-9110-1fa75033e0c1 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:08:47.685134 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f16ea1-17c7-42cb-91ca-0394e164fa6c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:08:47.687604 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f16ea1-17c7-42cb-91ca-0394e164fa6c service nova] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:08:47.698245 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 810/1601] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:08:47 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 15:08:47.710757 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f16ea1-17c7-42cb-91ca-0394e164fa6c service nova] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:08:47.713703 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:08:47.720069 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16f1607-7958-40e5-9110-1fa75033e0c1 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:08:47.725565 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f16ea1-17c7-42cb-91ca-0394e164fa6c service nova] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:08:47.727441 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e410d2-7e60-4eca-be81-97f58ccbe908 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:08:47.728375 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e410d2-7e60-4eca-be81-97f58ccbe908 admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:08:47.729000 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e410d2-7e60-4eca-be81-97f58ccbe908 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:08:47.729270 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e410d2-7e60-4eca-be81-97f58ccbe908 admin admin] PATH_INFO: `/v3/projects/68969c3e05e9433dbac7ff643427c5f4/users/8242513dc8c84bada9d2b647064cf06c/roles/d4088d1845a24fbd9bd8975ce0b1d3d0` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:08:47.730602 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c270f601-7c61-4f12-b05b-9716c72d2d8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:08:47.731776 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c270f601-7c61-4f12-b05b-9716c72d2d8f admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:08:47.732167 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c270f601-7c61-4f12-b05b-9716c72d2d8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:08:47.733954 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c270f601-7c61-4f12-b05b-9716c72d2d8f admin admin] PATH_INFO: `/v3/projects/87c12e0bbb554046a480ef02d9c78a14/users/b39b7ada271b455e9be595f23b9c5f3a/roles/d4088d1845a24fbd9bd8975ce0b1d3d0` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:08:47.747058 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e410d2-7e60-4eca-be81-97f58ccbe908 admin admin] RBAC: Authorizing `identity:create_grant(project_id=68969c3e05e9433dbac7ff643427c5f4, user_id=8242513dc8c84bada9d2b647064cf06c, role_id=d4088d1845a24fbd9bd8975ce0b1d3d0)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:08:47.750727 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e410d2-7e60-4eca-be81-97f58ccbe908 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:08:47.754850 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c270f601-7c61-4f12-b05b-9716c72d2d8f admin admin] RBAC: Authorizing `identity:create_grant(project_id=87c12e0bbb554046a480ef02d9c78a14, user_id=b39b7ada271b455e9be595f23b9c5f3a, role_id=d4088d1845a24fbd9bd8975ce0b1d3d0)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:08:47.759581 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c270f601-7c61-4f12-b05b-9716c72d2d8f admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:08:47.766077 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 792/1602] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 15:08:47 2022] PUT /identity/v3/projects/68969c3e05e9433dbac7ff643427c5f4/users/8242513dc8c84bada9d2b647064cf06c/roles/d4088d1845a24fbd9bd8975ce0b1d3d0 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:08:47.766997 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e410d2-7e60-4eca-be81-97f58ccbe908 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:08:47.791541 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 811/1603] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 15:08:47 2022] PUT /identity/v3/projects/87c12e0bbb554046a480ef02d9c78a14/users/b39b7ada271b455e9be595f23b9c5f3a/roles/d4088d1845a24fbd9bd8975ce0b1d3d0 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:08:47.791541 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c270f601-7c61-4f12-b05b-9716c72d2d8f admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:08:47.853918 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e410d2-7e60-4eca-be81-97f58ccbe908 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:08:47.857851 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:08:47.883188 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c270f601-7c61-4f12-b05b-9716c72d2d8f admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:08:47.888533 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe3b4da-effa-456a-a749-020ddac5b9e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:08:47.889516 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe3b4da-effa-456a-a749-020ddac5b9e3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:08:47.889808 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe3b4da-effa-456a-a749-020ddac5b9e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:08:47.890113 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe3b4da-effa-456a-a749-020ddac5b9e3 admin admin] PATH_INFO: `/v3/projects/d732418a4b2247e59ed96898b21cf6b1/users/64aeada36ebe460e9969f1f048651e6d/roles/d4088d1845a24fbd9bd8975ce0b1d3d0` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:08:47.907469 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe3b4da-effa-456a-a749-020ddac5b9e3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d732418a4b2247e59ed96898b21cf6b1, user_id=64aeada36ebe460e9969f1f048651e6d, role_id=d4088d1845a24fbd9bd8975ce0b1d3d0)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:08:47.910560 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe3b4da-effa-456a-a749-020ddac5b9e3 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:08:47.929252 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 812/1604] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 15:08:47 2022] PUT /identity/v3/projects/d732418a4b2247e59ed96898b21cf6b1/users/64aeada36ebe460e9969f1f048651e6d/roles/d4088d1845a24fbd9bd8975ce0b1d3d0 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:08:47.930838 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe3b4da-effa-456a-a749-020ddac5b9e3 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:08:47.937761 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ac31cb-174a-4e6a-8821-a340afa50a24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:08:47.938814 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ac31cb-174a-4e6a-8821-a340afa50a24 admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:08:47.939301 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ac31cb-174a-4e6a-8821-a340afa50a24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:08:47.939682 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ac31cb-174a-4e6a-8821-a340afa50a24 admin admin] PATH_INFO: `/v3/projects/9780d0cae62c4f5088060b530571ebb3/users/2ce5b5d5a12743bc80a4b191c24ff52f/roles/d4088d1845a24fbd9bd8975ce0b1d3d0` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:08:47.959248 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ac31cb-174a-4e6a-8821-a340afa50a24 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9780d0cae62c4f5088060b530571ebb3, user_id=2ce5b5d5a12743bc80a4b191c24ff52f, role_id=d4088d1845a24fbd9bd8975ce0b1d3d0)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:08:47.962142 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ac31cb-174a-4e6a-8821-a340afa50a24 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:08:47.982934 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 793/1605] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 15:08:47 2022] PUT /identity/v3/projects/9780d0cae62c4f5088060b530571ebb3/users/2ce5b5d5a12743bc80a4b191c24ff52f/roles/d4088d1845a24fbd9bd8975ce0b1d3d0 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:08:47.984510 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ac31cb-174a-4e6a-8821-a340afa50a24 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:08:48.017972 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe3b4da-effa-456a-a749-020ddac5b9e3 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:08:48.074327 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ac31cb-174a-4e6a-8821-a340afa50a24 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:08:48.080549 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6efa0b9-8e85-454c-bbec-bd999deafd25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:08:48.081705 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6efa0b9-8e85-454c-bbec-bd999deafd25 admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:08:48.082019 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6efa0b9-8e85-454c-bbec-bd999deafd25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:08:48.082309 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6efa0b9-8e85-454c-bbec-bd999deafd25 admin admin] PATH_INFO: `/v3/projects/0910873988084bc0becc79bb99febc96/users/e95b6425286f4235b9d1802acddac4b8/roles/d4088d1845a24fbd9bd8975ce0b1d3d0` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:08:48.090959 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf38185-deec-43c2-a62d-ba898d487948 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:08:48.091889 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf38185-deec-43c2-a62d-ba898d487948 admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:08:48.093500 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf38185-deec-43c2-a62d-ba898d487948 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:08:48.093500 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf38185-deec-43c2-a62d-ba898d487948 admin admin] PATH_INFO: `/v3/projects/ac537167367948ef88587242df4e921d/users/166bfe89e44847d7b380f33e2e28401d/roles/d4088d1845a24fbd9bd8975ce0b1d3d0` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:08:48.098330 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:08:48.102517 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6efa0b9-8e85-454c-bbec-bd999deafd25 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0910873988084bc0becc79bb99febc96, user_id=e95b6425286f4235b9d1802acddac4b8, role_id=d4088d1845a24fbd9bd8975ce0b1d3d0)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:08:48.106531 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6efa0b9-8e85-454c-bbec-bd999deafd25 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:08:48.114175 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf38185-deec-43c2-a62d-ba898d487948 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac537167367948ef88587242df4e921d, user_id=166bfe89e44847d7b380f33e2e28401d, role_id=d4088d1845a24fbd9bd8975ce0b1d3d0)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:08:48.116969 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf38185-deec-43c2-a62d-ba898d487948 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:08:48.131703 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 794/1606] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 15:08:47 2022] PUT /identity/v3/projects/0910873988084bc0becc79bb99febc96/users/e95b6425286f4235b9d1802acddac4b8/roles/d4088d1845a24fbd9bd8975ce0b1d3d0 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:08:48.133372 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208bf334-018d-4299-8c24-4a02cd9dbbdd None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:08:48.133743 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208bf334-018d-4299-8c24-4a02cd9dbbdd None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:08:48.134019 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208bf334-018d-4299-8c24-4a02cd9dbbdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:08:48.135604 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 813/1607] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 15:08:47 2022] PUT /identity/v3/projects/ac537167367948ef88587242df4e921d/users/166bfe89e44847d7b380f33e2e28401d/roles/d4088d1845a24fbd9bd8975ce0b1d3d0 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:08:48.137423 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9bf77c-8629-42ef-86b5-4b6453090f53 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:08:48.137879 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9bf77c-8629-42ef-86b5-4b6453090f53 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:08:48.138142 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9bf77c-8629-42ef-86b5-4b6453090f53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:08:48.157580 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-2c9bf77c-8629-42ef-86b5-4b6453090f53 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:08:48.159927 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-2c9bf77c-8629-42ef-86b5-4b6453090f53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:08:48.169916 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-208bf334-018d-4299-8c24-4a02cd9dbbdd None None] MFA Rules not processed for user `ff29b1cfe4c945a7a7aca87b79fc8075`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:08:48.172412 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-208bf334-018d-4299-8c24-4a02cd9dbbdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:08:48.229748 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:08:48.244527 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 814/1608] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:08:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:08:48.245988 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4c8e30-88d5-4ce8-82e4-e1e4670ea47d None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:08:48.246317 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4c8e30-88d5-4ce8-82e4-e1e4670ea47d None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:08:48.246706 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4c8e30-88d5-4ce8-82e4-e1e4670ea47d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:08:48.253244 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 795/1609] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:08:48 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:08:48.259223 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bf6c1a-d3b0-49fc-89af-31b2b59f4a66 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:08:48.259576 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bf6c1a-d3b0-49fc-89af-31b2b59f4a66 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:08:48.259908 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bf6c1a-d3b0-49fc-89af-31b2b59f4a66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:08:48.282122 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-5d4c8e30-88d5-4ce8-82e4-e1e4670ea47d None None] MFA Rules not processed for user `b39b7ada271b455e9be595f23b9c5f3a`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:08:48.283464 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-5d4c8e30-88d5-4ce8-82e4-e1e4670ea47d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:08:48.287102 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-87bf6c1a-d3b0-49fc-89af-31b2b59f4a66 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:08:48.288678 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-87bf6c1a-d3b0-49fc-89af-31b2b59f4a66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:08:48.306901 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 796/1610] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:08:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:08:48.306901 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057d0b15-1403-4118-ab56-b51896467c99 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:08:48.306901 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057d0b15-1403-4118-ab56-b51896467c99 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:08:48.306901 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057d0b15-1403-4118-ab56-b51896467c99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:08:48.326298 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-057d0b15-1403-4118-ab56-b51896467c99 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:08:48.327717 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-057d0b15-1403-4118-ab56-b51896467c99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:08:48.340872 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 797/1611] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:08:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:08:48.340872 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e047ce-326b-4ef4-86ce-b68e1c490695 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:08:48.340872 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e047ce-326b-4ef4-86ce-b68e1c490695 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:08:48.340872 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e047ce-326b-4ef4-86ce-b68e1c490695 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:08:48.341601 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 798/1612] 10.222.0.46 () {54 vars in 912 bytes} [Thu May 26 15:08:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 15:08:48.343311 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352bb95b-86fd-4527-8904-a9a9f9964f84 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:08:48.343672 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352bb95b-86fd-4527-8904-a9a9f9964f84 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:08:48.344072 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352bb95b-86fd-4527-8904-a9a9f9964f84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:08:48.367309 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-352bb95b-86fd-4527-8904-a9a9f9964f84 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:08:48.368577 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-352bb95b-86fd-4527-8904-a9a9f9964f84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:08:48.376744 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 815/1613] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:08:48 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:08:48.380770 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 799/1614] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:08:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:08:48.380770 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352bb95b-86fd-4527-8904-a9a9f9964f84 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:08:48.381086 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea0ce67-baf2-4ef1-b6da-feb3f22bd2dc None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:08:48.381086 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea0ce67-baf2-4ef1-b6da-feb3f22bd2dc None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:08:48.381086 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea0ce67-baf2-4ef1-b6da-feb3f22bd2dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:08:48.399479 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-4ea0ce67-baf2-4ef1-b6da-feb3f22bd2dc None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:08:48.400705 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-4ea0ce67-baf2-4ef1-b6da-feb3f22bd2dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:08:48.409940 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 816/1615] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:08:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:08:48.412744 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a635881-f99e-4dc0-8385-8a5fe998cc7c None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:08:48.413213 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a635881-f99e-4dc0-8385-8a5fe998cc7c None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:08:48.413997 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a635881-f99e-4dc0-8385-8a5fe998cc7c None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:08:48.416288 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 817/1616] 10.222.0.46 () {54 vars in 961 bytes} [Thu May 26 15:08:48 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 15:08:48.417256 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a635881-f99e-4dc0-8385-8a5fe998cc7c None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:08:48.447875 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352bb95b-86fd-4527-8904-a9a9f9964f84 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:08:48.453110 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c54dea-0f0c-4ab7-891f-3af5d178087e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0156a2f6f3794f479e76b85bdbbe5c26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:08:48.454017 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c54dea-0f0c-4ab7-891f-3af5d178087e service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:08:48.454355 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c54dea-0f0c-4ab7-891f-3af5d178087e service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:08:48.454820 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c54dea-0f0c-4ab7-891f-3af5d178087e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:08:48.474337 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c54dea-0f0c-4ab7-891f-3af5d178087e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:08:48.475965 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c54dea-0f0c-4ab7-891f-3af5d178087e service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:08:48.486263 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 800/1617] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:08:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 15:08:48.488006 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492eb94d-dc07-4560-9ce7-8e8438f81dc1 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:08:48.488158 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492eb94d-dc07-4560-9ce7-8e8438f81dc1 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:08:48.489138 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492eb94d-dc07-4560-9ce7-8e8438f81dc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:08:48.494483 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a635881-f99e-4dc0-8385-8a5fe998cc7c None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:08:48.500048 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fc2e99-2bea-46b9-ade1-00bf99425104 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0156a2f6f3794f479e76b85bdbbe5c26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:08:48.500801 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fc2e99-2bea-46b9-ade1-00bf99425104 service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:08:48.501211 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fc2e99-2bea-46b9-ade1-00bf99425104 service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:08:48.501534 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fc2e99-2bea-46b9-ade1-00bf99425104 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:08:48.509417 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-492eb94d-dc07-4560-9ce7-8e8438f81dc1 None None] MFA Rules not processed for user `d6639353c70449d1b3d10cd6487c3b00`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:08:48.511351 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-492eb94d-dc07-4560-9ce7-8e8438f81dc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:08:48.515470 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fc2e99-2bea-46b9-ade1-00bf99425104 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:08:48.517032 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fc2e99-2bea-46b9-ade1-00bf99425104 service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:08:48.531520 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 818/1618] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:08:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 15:08:48.532297 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fc2e99-2bea-46b9-ade1-00bf99425104 service neutron] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:08:48.533803 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-50fc2e99-2bea-46b9-ade1-00bf99425104 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:08:48.546754 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fc2e99-2bea-46b9-ade1-00bf99425104 service neutron] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:08:48.551192 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56eb5a96-dd9a-4de9-be14-fd55d4f41a3f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0156a2f6f3794f479e76b85bdbbe5c26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:08:48.552038 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56eb5a96-dd9a-4de9-be14-fd55d4f41a3f service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:08:48.552323 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56eb5a96-dd9a-4de9-be14-fd55d4f41a3f service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:08:48.552685 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56eb5a96-dd9a-4de9-be14-fd55d4f41a3f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:08:48.565851 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56eb5a96-dd9a-4de9-be14-fd55d4f41a3f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:08:48.567596 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56eb5a96-dd9a-4de9-be14-fd55d4f41a3f service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:08:48.581813 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 819/1619] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:08:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 15:08:48.582657 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56eb5a96-dd9a-4de9-be14-fd55d4f41a3f service neutron] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:08:48.595213 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56eb5a96-dd9a-4de9-be14-fd55d4f41a3f service neutron] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:08:48.597995 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:08:48.606858 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8240ebde-6e2a-4f2b-bb0d-d351fbfe05d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:08:48.607696 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8240ebde-6e2a-4f2b-bb0d-d351fbfe05d3 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:08:48.608247 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8240ebde-6e2a-4f2b-bb0d-d351fbfe05d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:08:48.610111 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8240ebde-6e2a-4f2b-bb0d-d351fbfe05d3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:08:48.611291 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8240ebde-6e2a-4f2b-bb0d-d351fbfe05d3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:08:48.616010 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8240ebde-6e2a-4f2b-bb0d-d351fbfe05d3 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:08:48.620633 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 801/1620] 10.222.0.46 () {58 vars in 1009 bytes} [Thu May 26 15:08:48 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:08:48.621404 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492eb94d-dc07-4560-9ce7-8e8438f81dc1 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:08:48.646485 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492eb94d-dc07-4560-9ce7-8e8438f81dc1 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:08:48.651863 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79da5d15-cc02-4ae5-944b-9b72fd55cd6e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0156a2f6f3794f479e76b85bdbbe5c26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:08:48.652148 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79da5d15-cc02-4ae5-944b-9b72fd55cd6e service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:08:48.652481 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79da5d15-cc02-4ae5-944b-9b72fd55cd6e service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:08:48.652766 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79da5d15-cc02-4ae5-944b-9b72fd55cd6e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:08:48.667173 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79da5d15-cc02-4ae5-944b-9b72fd55cd6e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:08:48.668696 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79da5d15-cc02-4ae5-944b-9b72fd55cd6e service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:08:48.679797 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 820/1621] 10.222.0.46 () {60 vars in 1152 bytes} [Thu May 26 15:08:48 2022] POST /identity/v3/projects => generated 403 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 15:08:48.682823 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50812242-7c38-4ba3-aee8-5845c96f26e0 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:08:48.683518 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50812242-7c38-4ba3-aee8-5845c96f26e0 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:08:48.683869 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 802/1622] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:08:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 15:08:48.684457 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50812242-7c38-4ba3-aee8-5845c96f26e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:08:48.685302 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79da5d15-cc02-4ae5-944b-9b72fd55cd6e service neutron] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:08:48.699085 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79da5d15-cc02-4ae5-944b-9b72fd55cd6e service neutron] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:08:48.723191 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-50812242-7c38-4ba3-aee8-5845c96f26e0 None None] MFA Rules not processed for user `d6639353c70449d1b3d10cd6487c3b00`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:08:48.724939 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-50812242-7c38-4ba3-aee8-5845c96f26e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:08:48.791709 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d37555-1ade-4b07-bbc3-39f52c7a9799 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0156a2f6f3794f479e76b85bdbbe5c26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:08:48.792543 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d37555-1ade-4b07-bbc3-39f52c7a9799 service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:08:48.792829 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d37555-1ade-4b07-bbc3-39f52c7a9799 service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:08:48.793104 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d37555-1ade-4b07-bbc3-39f52c7a9799 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:08:48.799389 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:08:48.844917 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 821/1623] 10.222.0.46 () {58 vars in 1058 bytes} [Thu May 26 15:08:48 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:08:48.846215 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50812242-7c38-4ba3-aee8-5845c96f26e0 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:08:48.893614 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d37555-1ade-4b07-bbc3-39f52c7a9799 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:08:48.897084 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d37555-1ade-4b07-bbc3-39f52c7a9799 service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:08:48.913318 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 803/1624] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:08:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 15:08:48.913318 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d37555-1ade-4b07-bbc3-39f52c7a9799 service neutron] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:08:48.927939 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d37555-1ade-4b07-bbc3-39f52c7a9799 service neutron] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:08:48.933057 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67ae88b-9068-4141-8885-41cfee2a9b09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:08:48.933841 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67ae88b-9068-4141-8885-41cfee2a9b09 admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:08:48.934135 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67ae88b-9068-4141-8885-41cfee2a9b09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:08:48.934406 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f67ae88b-9068-4141-8885-41cfee2a9b09 admin admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:08:48.935105 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67ae88b-9068-4141-8885-41cfee2a9b09 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:08:48.938640 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f67ae88b-9068-4141-8885-41cfee2a9b09 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:08:48.975423 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 804/1625] 10.222.0.46 () {60 vars in 1143 bytes} [Thu May 26 15:08:48 2022] POST /identity/v3/users => generated 308 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 15:08:48.975423 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67ae88b-9068-4141-8885-41cfee2a9b09 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:08:48.975642 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50812242-7c38-4ba3-aee8-5845c96f26e0 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:08:48.979563 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cd0f9f-997b-49aa-a094-9d0e85912be6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5b5b35614a547d783e6e77ddb8c1c03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:08:48.991005 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cd0f9f-997b-49aa-a094-9d0e85912be6 service cinder] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:08:48.991718 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f67ae88b-9068-4141-8885-41cfee2a9b09 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:08:48.993664 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cd0f9f-997b-49aa-a094-9d0e85912be6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:08:48.995136 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cd0f9f-997b-49aa-a094-9d0e85912be6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:08:49.003164 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ed3926-ff2e-4a71-91a4-bade682b6865 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6639353c70449d1b3d10cd6487c3b00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:08:49.004275 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ed3926-ff2e-4a71-91a4-bade682b6865 service nova] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:08:49.004656 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ed3926-ff2e-4a71-91a4-bade682b6865 service nova] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:08:49.005252 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ed3926-ff2e-4a71-91a4-bade682b6865 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:08:49.024283 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84cd0f9f-997b-49aa-a094-9d0e85912be6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:08:49.026167 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84cd0f9f-997b-49aa-a094-9d0e85912be6 service cinder] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:08:49.042412 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 822/1626] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:08:48 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 15:08:49.043410 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cd0f9f-997b-49aa-a094-9d0e85912be6 service cinder] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:08:49.056160 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cd0f9f-997b-49aa-a094-9d0e85912be6 service cinder] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:08:49.066955 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d58cd2-7400-4841-bfbf-8d1a9f39c9cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:08:49.071097 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d58cd2-7400-4841-bfbf-8d1a9f39c9cc admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:08:49.071604 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d58cd2-7400-4841-bfbf-8d1a9f39c9cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:08:49.073986 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d58cd2-7400-4841-bfbf-8d1a9f39c9cc admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:08:49.075012 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d58cd2-7400-4841-bfbf-8d1a9f39c9cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:08:49.079800 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d58cd2-7400-4841-bfbf-8d1a9f39c9cc admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:08:49.113147 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 823/1627] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 15:08:49 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 15:08:49.113368 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ed3926-ff2e-4a71-91a4-bade682b6865 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:08:49.113368 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ed3926-ff2e-4a71-91a4-bade682b6865 service nova] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:08:49.114714 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d58cd2-7400-4841-bfbf-8d1a9f39c9cc admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:08:49.138311 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 805/1628] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:08:48 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 15:08:49.149821 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d58cd2-7400-4841-bfbf-8d1a9f39c9cc admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:08:49.182063 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9d53d7-d00a-4f4c-b77e-51bb644c1935 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:08:49.183216 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9d53d7-d00a-4f4c-b77e-51bb644c1935 admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:08:49.183637 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9d53d7-d00a-4f4c-b77e-51bb644c1935 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:08:49.184011 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9d53d7-d00a-4f4c-b77e-51bb644c1935 admin admin] PATH_INFO: `/v3/projects/18277aa7a5374fed932e84717059d8ac/users/b04b122515244068a9c83c62fca584a8/roles/a2e551773cbd4df084020676865b8222` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:08:49.243551 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9d53d7-d00a-4f4c-b77e-51bb644c1935 admin admin] RBAC: Authorizing `identity:create_grant(project_id=18277aa7a5374fed932e84717059d8ac, user_id=b04b122515244068a9c83c62fca584a8, role_id=a2e551773cbd4df084020676865b8222)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:08:49.247510 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9d53d7-d00a-4f4c-b77e-51bb644c1935 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:08:49.279012 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 824/1629] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 15:08:49 2022] PUT /identity/v3/projects/18277aa7a5374fed932e84717059d8ac/users/b04b122515244068a9c83c62fca584a8/roles/a2e551773cbd4df084020676865b8222 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:08:49.283386 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ed3926-ff2e-4a71-91a4-bade682b6865 service nova] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:08:49.303485 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ed3926-ff2e-4a71-91a4-bade682b6865 service nova] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:08:49.340094 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:08:49.516135 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3441aeee-cd11-455b-a1fc-d222c690229e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:08:49.516958 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3441aeee-cd11-455b-a1fc-d222c690229e admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:08:49.517318 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3441aeee-cd11-455b-a1fc-d222c690229e admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:08:49.517699 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3441aeee-cd11-455b-a1fc-d222c690229e admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:08:49.518629 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3441aeee-cd11-455b-a1fc-d222c690229e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:08:49.520782 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3441aeee-cd11-455b-a1fc-d222c690229e admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:08:49.536524 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 806/1630] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 15:08:49 2022] GET /identity/v3/roles => generated 1420 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 15:08:49.540378 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9d53d7-d00a-4f4c-b77e-51bb644c1935 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:08:49.570211 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9d53d7-d00a-4f4c-b77e-51bb644c1935 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:08:49.595704 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-932c0ebd-7436-4e75-8cda-152a67dfcac4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:08:49.596740 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932c0ebd-7436-4e75-8cda-152a67dfcac4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:08:49.597079 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932c0ebd-7436-4e75-8cda-152a67dfcac4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:08:49.597436 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-932c0ebd-7436-4e75-8cda-152a67dfcac4 admin admin] PATH_INFO: `/v3/projects/18277aa7a5374fed932e84717059d8ac/users/b04b122515244068a9c83c62fca584a8/roles/a2e551773cbd4df084020676865b8222` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:08:49.608861 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932c0ebd-7436-4e75-8cda-152a67dfcac4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=18277aa7a5374fed932e84717059d8ac, user_id=b04b122515244068a9c83c62fca584a8, role_id=a2e551773cbd4df084020676865b8222)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:08:49.615826 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-932c0ebd-7436-4e75-8cda-152a67dfcac4 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:08:49.681619 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 825/1631] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 15:08:49 2022] PUT /identity/v3/projects/18277aa7a5374fed932e84717059d8ac/users/b04b122515244068a9c83c62fca584a8/roles/a2e551773cbd4df084020676865b8222 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:08:49.690652 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2de695-3ffe-4de8-bba9-78ee2234bf87 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:08:49.690954 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2de695-3ffe-4de8-bba9-78ee2234bf87 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:08:49.691282 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2de695-3ffe-4de8-bba9-78ee2234bf87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:08:49.832180 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-5b2de695-3ffe-4de8-bba9-78ee2234bf87 None None] MFA Rules not processed for user `b04b122515244068a9c83c62fca584a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:08:49.833938 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-5b2de695-3ffe-4de8-bba9-78ee2234bf87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:08:50.063743 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 807/1632] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:08:49 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 374 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:08:50.074217 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0515dde9-5201-42c9-8692-fcce864b47aa None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:08:50.074651 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0515dde9-5201-42c9-8692-fcce864b47aa None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:08:50.075339 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0515dde9-5201-42c9-8692-fcce864b47aa None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:08:50.079787 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 826/1633] 10.222.0.46 () {54 vars in 912 bytes} [Thu May 26 15:08:50 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 15:08:50.090575 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0515dde9-5201-42c9-8692-fcce864b47aa None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:08:50.098192 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0754c340-8fb0-4119-b1a1-33047e304100 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:08:50.100941 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0754c340-8fb0-4119-b1a1-33047e304100 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:08:50.101898 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0754c340-8fb0-4119-b1a1-33047e304100 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:08:50.200513 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-0754c340-8fb0-4119-b1a1-33047e304100 None None] MFA Rules not processed for user `d6639353c70449d1b3d10cd6487c3b00`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:08:50.213505 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-0754c340-8fb0-4119-b1a1-33047e304100 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:08:50.248685 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0515dde9-5201-42c9-8692-fcce864b47aa None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:08:50.254723 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47550f60-b593-4a4d-898a-152df5247dd0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5b5b35614a547d783e6e77ddb8c1c03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:08:50.255893 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47550f60-b593-4a4d-898a-152df5247dd0 service cinder] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:08:50.256321 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47550f60-b593-4a4d-898a-152df5247dd0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:08:50.256835 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47550f60-b593-4a4d-898a-152df5247dd0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:08:50.270420 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47550f60-b593-4a4d-898a-152df5247dd0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:08:50.272035 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47550f60-b593-4a4d-898a-152df5247dd0 service cinder] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:08:50.284223 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 827/1634] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:08:50 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 15:08:50.318798 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a96a338-a41a-4be4-af04-93e89f340341 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:08:50.319268 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a96a338-a41a-4be4-af04-93e89f340341 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:08:50.319718 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a96a338-a41a-4be4-af04-93e89f340341 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:08:50.374962 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-6a96a338-a41a-4be4-af04-93e89f340341 None None] MFA Rules not processed for user `166bfe89e44847d7b380f33e2e28401d`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:08:50.376556 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-6a96a338-a41a-4be4-af04-93e89f340341 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:08:50.413298 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 808/1635] 10.222.0.46 () {58 vars in 1009 bytes} [Thu May 26 15:08:50 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 327 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:08:50.501133 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0754c340-8fb0-4119-b1a1-33047e304100 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:08:50.502253 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:08:50.523529 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0754c340-8fb0-4119-b1a1-33047e304100 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:08:50.538014 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c40355-365b-4f07-b94e-59b6c638d3a2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5b5b35614a547d783e6e77ddb8c1c03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:08:50.541310 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c40355-365b-4f07-b94e-59b6c638d3a2 service cinder] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:08:50.541706 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c40355-365b-4f07-b94e-59b6c638d3a2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:08:50.542026 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c40355-365b-4f07-b94e-59b6c638d3a2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:08:50.579512 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c40355-365b-4f07-b94e-59b6c638d3a2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:08:50.581132 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c40355-365b-4f07-b94e-59b6c638d3a2 service cinder] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:08:50.627045 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 809/1636] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:08:50 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 15:08:50.693968 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 828/1637] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:08:50 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 377 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:08:50.712618 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7582932-c23f-4fcf-bcc9-83350d434392 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:08:50.713551 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7582932-c23f-4fcf-bcc9-83350d434392 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:08:50.714711 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7582932-c23f-4fcf-bcc9-83350d434392 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:08:50.841996 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-a7582932-c23f-4fcf-bcc9-83350d434392 None None] MFA Rules not processed for user `166bfe89e44847d7b380f33e2e28401d`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:08:50.846251 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-a7582932-c23f-4fcf-bcc9-83350d434392 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:08:50.880245 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 810/1638] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:08:50 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:08:50.941858 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7582932-c23f-4fcf-bcc9-83350d434392 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:08:50.943105 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a96a338-a41a-4be4-af04-93e89f340341 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:08:50.974947 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7582932-c23f-4fcf-bcc9-83350d434392 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:08:51.008568 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dccfdc6-2b0e-403c-b536-9b0957992188 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6639353c70449d1b3d10cd6487c3b00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:08:51.009414 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dccfdc6-2b0e-403c-b536-9b0957992188 service nova] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:08:51.009797 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dccfdc6-2b0e-403c-b536-9b0957992188 service nova] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:08:51.010145 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dccfdc6-2b0e-403c-b536-9b0957992188 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:08:51.023138 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:08:51.033338 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dccfdc6-2b0e-403c-b536-9b0957992188 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:08:51.048171 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dccfdc6-2b0e-403c-b536-9b0957992188 service nova] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:08:51.077778 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 811/1639] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:08:50 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 15:08:51.172314 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a96a338-a41a-4be4-af04-93e89f340341 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:08:51.181072 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846829b7-8bf4-48b2-90d9-4c68327d6da7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:08:51.182983 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846829b7-8bf4-48b2-90d9-4c68327d6da7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:08:51.183675 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846829b7-8bf4-48b2-90d9-4c68327d6da7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:08:51.184715 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846829b7-8bf4-48b2-90d9-4c68327d6da7 admin admin] PATH_INFO: `/v3/users/b39b7ada271b455e9be595f23b9c5f3a` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:08:51.188948 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846829b7-8bf4-48b2-90d9-4c68327d6da7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b39b7ada271b455e9be595f23b9c5f3a)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:08:51.193793 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846829b7-8bf4-48b2-90d9-4c68327d6da7 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:08:51.306244 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dccfdc6-2b0e-403c-b536-9b0957992188 service nova] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:08:51.330360 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dccfdc6-2b0e-403c-b536-9b0957992188 service nova] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:08:51.336749 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e146ff1b-e9d9-4980-bee4-17d79a091b34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:08:51.338192 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e146ff1b-e9d9-4980-bee4-17d79a091b34 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:08:51.339343 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e146ff1b-e9d9-4980-bee4-17d79a091b34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:08:51.339904 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e146ff1b-e9d9-4980-bee4-17d79a091b34 admin admin] PATH_INFO: `/v3/users/ff29b1cfe4c945a7a7aca87b79fc8075` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:08:51.341848 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e146ff1b-e9d9-4980-bee4-17d79a091b34 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ff29b1cfe4c945a7a7aca87b79fc8075)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:08:51.343398 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: WARNING py.warnings [None req-e146ff1b-e9d9-4980-bee4-17d79a091b34 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 26 15:08:51.343398 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: warnings.warn(msg) May 26 15:08:51.343398 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: May 26 15:08:51.345193 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e146ff1b-e9d9-4980-bee4-17d79a091b34 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:08:51.362209 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-846829b7-8bf4-48b2-90d9-4c68327d6da7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b39b7ada271b455e9be595f23b9c5f3a'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:08:51.380069 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-846829b7-8bf4-48b2-90d9-4c68327d6da7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b39b7ada271b455e9be595f23b9c5f3a'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:08:51.381854 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-846829b7-8bf4-48b2-90d9-4c68327d6da7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b39b7ada271b455e9be595f23b9c5f3a'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:08:51.426918 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-846829b7-8bf4-48b2-90d9-4c68327d6da7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b39b7ada271b455e9be595f23b9c5f3a'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:08:51.448337 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-846829b7-8bf4-48b2-90d9-4c68327d6da7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b39b7ada271b455e9be595f23b9c5f3a'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:08:51.457152 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 829/1640] 10.222.0.46 () {58 vars in 1223 bytes} [Thu May 26 15:08:50 2022] DELETE /identity/v3/users/b39b7ada271b455e9be595f23b9c5f3a => generated 0 bytes in 525 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:08:51.462737 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b7c246-f2b2-4e39-9e7c-d8275338eedd None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:08:51.463026 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b7c246-f2b2-4e39-9e7c-d8275338eedd None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:08:51.463583 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b7c246-f2b2-4e39-9e7c-d8275338eedd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:08:51.495241 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-e146ff1b-e9d9-4980-bee4-17d79a091b34 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ff29b1cfe4c945a7a7aca87b79fc8075'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:08:51.509170 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-e146ff1b-e9d9-4980-bee4-17d79a091b34 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ff29b1cfe4c945a7a7aca87b79fc8075'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:08:51.509874 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-e146ff1b-e9d9-4980-bee4-17d79a091b34 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ff29b1cfe4c945a7a7aca87b79fc8075'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:08:51.510678 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-c3b7c246-f2b2-4e39-9e7c-d8275338eedd None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:08:51.512815 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-c3b7c246-f2b2-4e39-9e7c-d8275338eedd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:08:51.542252 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-e146ff1b-e9d9-4980-bee4-17d79a091b34 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ff29b1cfe4c945a7a7aca87b79fc8075'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:08:51.559816 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-e146ff1b-e9d9-4980-bee4-17d79a091b34 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ff29b1cfe4c945a7a7aca87b79fc8075'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:08:51.578563 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 812/1641] 10.222.0.46 () {58 vars in 1223 bytes} [Thu May 26 15:08:51 2022] DELETE /identity/v3/users/ff29b1cfe4c945a7a7aca87b79fc8075 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 15:08:51.580860 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3a6485-137c-4f5d-83ef-6504603c79e6 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:08:51.581160 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3a6485-137c-4f5d-83ef-6504603c79e6 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:08:51.581526 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3a6485-137c-4f5d-83ef-6504603c79e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:08:51.616402 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:08:51.631463 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-7f3a6485-137c-4f5d-83ef-6504603c79e6 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:08:51.638566 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-7f3a6485-137c-4f5d-83ef-6504603c79e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:08:51.656871 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 830/1642] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:08:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 15:08:51.707950 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b7c246-f2b2-4e39-9e7c-d8275338eedd None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:08:51.709618 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-c3b7c246-f2b2-4e39-9e7c-d8275338eedd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:08:51.778083 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 813/1643] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:08:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:08:51.801986 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3a6485-137c-4f5d-83ef-6504603c79e6 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:08:51.804333 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-7f3a6485-137c-4f5d-83ef-6504603c79e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:08:51.828181 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b7c246-f2b2-4e39-9e7c-d8275338eedd None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:08:51.923029 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f57379-b31e-4417-abc8-48be52ae584e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0156a2f6f3794f479e76b85bdbbe5c26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:08:51.924202 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f57379-b31e-4417-abc8-48be52ae584e service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:08:51.924827 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f57379-b31e-4417-abc8-48be52ae584e service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:08:51.925216 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f57379-b31e-4417-abc8-48be52ae584e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:08:51.938985 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f57379-b31e-4417-abc8-48be52ae584e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:08:51.940677 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f57379-b31e-4417-abc8-48be52ae584e service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:08:51.953106 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 831/1644] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:08:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 246 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 15:08:51.955251 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac69590-5d6c-423c-a1ad-900205c85906 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:08:51.956062 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac69590-5d6c-423c-a1ad-900205c85906 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:08:51.956822 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac69590-5d6c-423c-a1ad-900205c85906 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:08:51.973328 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3a6485-137c-4f5d-83ef-6504603c79e6 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:08:51.986269 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397b137c-b5c1-4330-83b8-e5e43d772722 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0156a2f6f3794f479e76b85bdbbe5c26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:08:51.987165 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397b137c-b5c1-4330-83b8-e5e43d772722 service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:08:51.987461 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397b137c-b5c1-4330-83b8-e5e43d772722 service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:08:51.987865 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397b137c-b5c1-4330-83b8-e5e43d772722 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:08:52.016594 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397b137c-b5c1-4330-83b8-e5e43d772722 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:08:52.019629 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397b137c-b5c1-4330-83b8-e5e43d772722 service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:08:52.040450 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-6ac69590-5d6c-423c-a1ad-900205c85906 None None] MFA Rules not processed for user `8242513dc8c84bada9d2b647064cf06c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:08:52.043088 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-6ac69590-5d6c-423c-a1ad-900205c85906 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:08:52.052081 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 814/1645] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:08:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 251 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 15:08:52.208368 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 832/1646] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:08:51 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:08:52.214157 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397b137c-b5c1-4330-83b8-e5e43d772722 service neutron] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:08:52.216775 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-397b137c-b5c1-4330-83b8-e5e43d772722 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:08:52.257133 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397b137c-b5c1-4330-83b8-e5e43d772722 service neutron] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:08:52.273186 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b45c4d0-fe23-4148-9a73-31239a48c571 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:08:52.275056 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b45c4d0-fe23-4148-9a73-31239a48c571 admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:08:52.277546 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b45c4d0-fe23-4148-9a73-31239a48c571 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:08:52.278368 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b45c4d0-fe23-4148-9a73-31239a48c571 admin admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:08:52.279770 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b45c4d0-fe23-4148-9a73-31239a48c571 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:08:52.289069 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b45c4d0-fe23-4148-9a73-31239a48c571 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:08:52.308697 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:08:52.377091 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 815/1647] 10.222.0.46 () {60 vars in 1152 bytes} [Thu May 26 15:08:52 2022] POST /identity/v3/projects => generated 405 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 15:08:52.381428 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac69590-5d6c-423c-a1ad-900205c85906 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:08:52.419870 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac69590-5d6c-423c-a1ad-900205c85906 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:08:52.547682 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:08:52.583139 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5282dcd7-6d13-414c-a8ac-49e47a137f95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:08:52.584149 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5282dcd7-6d13-414c-a8ac-49e47a137f95 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:08:52.584883 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5282dcd7-6d13-414c-a8ac-49e47a137f95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:08:52.585278 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5282dcd7-6d13-414c-a8ac-49e47a137f95 admin admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:08:52.586571 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5282dcd7-6d13-414c-a8ac-49e47a137f95 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:08:52.589436 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5282dcd7-6d13-414c-a8ac-49e47a137f95 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:08:52.621530 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 833/1648] 10.222.0.46 () {60 vars in 1143 bytes} [Thu May 26 15:08:52 2022] POST /identity/v3/users => generated 309 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 15:08:52.626600 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b45c4d0-fe23-4148-9a73-31239a48c571 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:08:52.642974 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b45c4d0-fe23-4148-9a73-31239a48c571 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:08:52.656978 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cebca67-b706-4179-bd50-de00cf116a80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:08:52.658297 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cebca67-b706-4179-bd50-de00cf116a80 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:08:52.660950 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cebca67-b706-4179-bd50-de00cf116a80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:08:52.663096 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cebca67-b706-4179-bd50-de00cf116a80 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:08:52.667313 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cebca67-b706-4179-bd50-de00cf116a80 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:08:52.673598 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cebca67-b706-4179-bd50-de00cf116a80 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:08:52.711885 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 816/1649] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 15:08:52 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 15:08:52.714737 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5282dcd7-6d13-414c-a8ac-49e47a137f95 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:08:52.789134 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5282dcd7-6d13-414c-a8ac-49e47a137f95 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:08:52.815660 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adf9fef-8ca4-4911-9c99-2f85d3268767 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:08:52.820545 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adf9fef-8ca4-4911-9c99-2f85d3268767 admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:08:52.820917 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adf9fef-8ca4-4911-9c99-2f85d3268767 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:08:52.821124 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adf9fef-8ca4-4911-9c99-2f85d3268767 admin admin] PATH_INFO: `/v3/projects/e4e82e4c96364f3e85f5c8673fdf5d91/users/fe5f60f71d5649bc8449be91da9127e9/roles/a2e551773cbd4df084020676865b8222` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:08:52.880799 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0adf9fef-8ca4-4911-9c99-2f85d3268767 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e4e82e4c96364f3e85f5c8673fdf5d91, user_id=fe5f60f71d5649bc8449be91da9127e9, role_id=a2e551773cbd4df084020676865b8222)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:08:52.884574 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0adf9fef-8ca4-4911-9c99-2f85d3268767 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:08:52.909379 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 834/1650] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 15:08:52 2022] PUT /identity/v3/projects/e4e82e4c96364f3e85f5c8673fdf5d91/users/fe5f60f71d5649bc8449be91da9127e9/roles/a2e551773cbd4df084020676865b8222 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:08:52.914964 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cebca67-b706-4179-bd50-de00cf116a80 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:08:52.948914 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cebca67-b706-4179-bd50-de00cf116a80 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:08:53.124533 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7cec2b-4f07-4d4c-b8fe-8599ca3be1a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:08:53.125359 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7cec2b-4f07-4d4c-b8fe-8599ca3be1a6 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:08:53.126255 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7cec2b-4f07-4d4c-b8fe-8599ca3be1a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:08:53.126255 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7cec2b-4f07-4d4c-b8fe-8599ca3be1a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:08:53.127198 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7cec2b-4f07-4d4c-b8fe-8599ca3be1a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:08:53.129262 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7cec2b-4f07-4d4c-b8fe-8599ca3be1a6 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:08:53.146226 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c93e26-444d-49a1-88e7-4dcd05f723f2 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:08:53.146461 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c93e26-444d-49a1-88e7-4dcd05f723f2 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:08:53.146888 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c93e26-444d-49a1-88e7-4dcd05f723f2 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:08:53.149407 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 835/1651] 10.222.0.46 () {54 vars in 912 bytes} [Thu May 26 15:08:53 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 15:08:53.159848 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85766b1-f3a3-49e6-95ba-fc13e83383a3 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:08:53.160459 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85766b1-f3a3-49e6-95ba-fc13e83383a3 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:08:53.160907 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85766b1-f3a3-49e6-95ba-fc13e83383a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:08:53.161881 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 817/1652] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 15:08:52 2022] GET /identity/v3/roles => generated 1420 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 15:08:53.165728 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7cec2b-4f07-4d4c-b8fe-8599ca3be1a6 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:08:53.181824 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7cec2b-4f07-4d4c-b8fe-8599ca3be1a6 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:08:53.188359 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef57a18-8df7-41f2-af80-363f6f210500 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:08:53.190905 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef57a18-8df7-41f2-af80-363f6f210500 admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:08:53.191689 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef57a18-8df7-41f2-af80-363f6f210500 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:08:53.192202 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef57a18-8df7-41f2-af80-363f6f210500 admin admin] PATH_INFO: `/v3/projects/e4e82e4c96364f3e85f5c8673fdf5d91/users/fe5f60f71d5649bc8449be91da9127e9/roles/a2e551773cbd4df084020676865b8222` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:08:53.195940 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef57a18-8df7-41f2-af80-363f6f210500 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e4e82e4c96364f3e85f5c8673fdf5d91, user_id=fe5f60f71d5649bc8449be91da9127e9, role_id=a2e551773cbd4df084020676865b8222)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:08:53.200194 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef57a18-8df7-41f2-af80-363f6f210500 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:08:53.217588 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 818/1653] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 15:08:53 2022] PUT /identity/v3/projects/e4e82e4c96364f3e85f5c8673fdf5d91/users/fe5f60f71d5649bc8449be91da9127e9/roles/a2e551773cbd4df084020676865b8222 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:08:53.259577 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-e85766b1-f3a3-49e6-95ba-fc13e83383a3 None None] MFA Rules not processed for user `0156a2f6f3794f479e76b85bdbbe5c26`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:08:53.261142 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-e85766b1-f3a3-49e6-95ba-fc13e83383a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:08:53.402980 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 836/1654] 10.222.0.46 () {58 vars in 1009 bytes} [Thu May 26 15:08:53 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:08:53.489895 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef57a18-8df7-41f2-af80-363f6f210500 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:08:53.524055 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef57a18-8df7-41f2-af80-363f6f210500 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:08:53.539529 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c49a44-b5e9-40ef-b1d6-687cfe87f97c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0156a2f6f3794f479e76b85bdbbe5c26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:08:53.541305 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c49a44-b5e9-40ef-b1d6-687cfe87f97c service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:08:53.541826 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c49a44-b5e9-40ef-b1d6-687cfe87f97c service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:08:53.542656 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c49a44-b5e9-40ef-b1d6-687cfe87f97c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:08:53.559478 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c49a44-b5e9-40ef-b1d6-687cfe87f97c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:08:53.563611 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c49a44-b5e9-40ef-b1d6-687cfe87f97c service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:08:53.578267 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 819/1655] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:08:53 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 15:08:54.924152 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85766b1-f3a3-49e6-95ba-fc13e83383a3 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:08:54.926182 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-e85766b1-f3a3-49e6-95ba-fc13e83383a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:08:54.989490 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c49a44-b5e9-40ef-b1d6-687cfe87f97c service neutron] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:08:54.994015 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-a4c49a44-b5e9-40ef-b1d6-687cfe87f97c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:08:55.143969 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85766b1-f3a3-49e6-95ba-fc13e83383a3 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:08:55.159511 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962dce14-ec3d-4c22-a527-800fd9c39dc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:08:55.161563 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962dce14-ec3d-4c22-a527-800fd9c39dc8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:08:55.165615 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962dce14-ec3d-4c22-a527-800fd9c39dc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:08:55.167028 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962dce14-ec3d-4c22-a527-800fd9c39dc8 admin admin] PATH_INFO: `/v3/projects/612679bca8894138b317a0c19d8b2069` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:08:55.180204 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962dce14-ec3d-4c22-a527-800fd9c39dc8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=612679bca8894138b317a0c19d8b2069)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:08:55.195571 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: WARNING py.warnings [None req-962dce14-ec3d-4c22-a527-800fd9c39dc8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 26 15:08:55.195571 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: warnings.warn(msg) May 26 15:08:55.195571 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: May 26 15:08:55.197236 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962dce14-ec3d-4c22-a527-800fd9c39dc8 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:08:55.268195 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c49a44-b5e9-40ef-b1d6-687cfe87f97c service neutron] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:08:55.277869 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf635a6-c1c5-4764-8e2c-4688714b14d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:08:55.278971 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf635a6-c1c5-4764-8e2c-4688714b14d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:08:55.279358 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf635a6-c1c5-4764-8e2c-4688714b14d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:08:55.279778 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf635a6-c1c5-4764-8e2c-4688714b14d2 admin admin] PATH_INFO: `/v3/projects/87c12e0bbb554046a480ef02d9c78a14` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:08:55.285714 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf635a6-c1c5-4764-8e2c-4688714b14d2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=87c12e0bbb554046a480ef02d9c78a14)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:08:55.287504 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf635a6-c1c5-4764-8e2c-4688714b14d2 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:08:55.311355 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:08:55.365722 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-962dce14-ec3d-4c22-a527-800fd9c39dc8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '612679bca8894138b317a0c19d8b2069', 'request_id': 'req-962dce14-ec3d-4c22-a527-800fd9c39dc8'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:08:55.392802 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-962dce14-ec3d-4c22-a527-800fd9c39dc8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '612679bca8894138b317a0c19d8b2069', 'request_id': 'req-962dce14-ec3d-4c22-a527-800fd9c39dc8'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:08:55.413196 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-962dce14-ec3d-4c22-a527-800fd9c39dc8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '612679bca8894138b317a0c19d8b2069', 'request_id': 'req-962dce14-ec3d-4c22-a527-800fd9c39dc8'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:08:55.453183 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=94,keystone:INSERT=2,keystone:DELETE=4 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:08:55.456140 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-bbf635a6-c1c5-4764-8e2c-4688714b14d2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '87c12e0bbb554046a480ef02d9c78a14', 'request_id': 'req-bbf635a6-c1c5-4764-8e2c-4688714b14d2'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:08:55.514891 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-962dce14-ec3d-4c22-a527-800fd9c39dc8 admin admin] The token cache is being invalidate because project 612679bca8894138b317a0c19d8b2069 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 15:08:55.515324 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-bbf635a6-c1c5-4764-8e2c-4688714b14d2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '87c12e0bbb554046a480ef02d9c78a14', 'request_id': 'req-bbf635a6-c1c5-4764-8e2c-4688714b14d2'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:08:55.515892 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-962dce14-ec3d-4c22-a527-800fd9c39dc8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:08:55.517354 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-962dce14-ec3d-4c22-a527-800fd9c39dc8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:08:55.520319 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 837/1656] 10.222.0.46 () {58 vars in 1232 bytes} [Thu May 26 15:08:54 2022] DELETE /identity/v3/projects/612679bca8894138b317a0c19d8b2069 => generated 0 bytes in 596 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:08:55.525848 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039d446c-3742-4680-9263-cb7ba27b6358 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:08:55.526815 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039d446c-3742-4680-9263-cb7ba27b6358 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:08:55.527296 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039d446c-3742-4680-9263-cb7ba27b6358 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:08:55.571629 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-bbf635a6-c1c5-4764-8e2c-4688714b14d2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '87c12e0bbb554046a480ef02d9c78a14', 'request_id': 'req-bbf635a6-c1c5-4764-8e2c-4688714b14d2'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:08:55.613765 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-039d446c-3742-4680-9263-cb7ba27b6358 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:08:55.616050 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-039d446c-3742-4680-9263-cb7ba27b6358 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:08:55.675646 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-bbf635a6-c1c5-4764-8e2c-4688714b14d2 admin admin] The token cache is being invalidate because project 87c12e0bbb554046a480ef02d9c78a14 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 15:08:55.676295 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-bbf635a6-c1c5-4764-8e2c-4688714b14d2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:08:55.677512 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-bbf635a6-c1c5-4764-8e2c-4688714b14d2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:08:55.681517 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 820/1657] 10.222.0.46 () {58 vars in 1232 bytes} [Thu May 26 15:08:54 2022] DELETE /identity/v3/projects/87c12e0bbb554046a480ef02d9c78a14 => generated 0 bytes in 693 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:08:55.683472 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efb1245-8a87-4add-bca0-c501279c35b9 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:08:55.684531 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efb1245-8a87-4add-bca0-c501279c35b9 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:08:55.685024 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efb1245-8a87-4add-bca0-c501279c35b9 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:08:55.693920 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 821/1658] 10.222.0.46 () {54 vars in 960 bytes} [Thu May 26 15:08:55 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 15:08:55.695495 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efb1245-8a87-4add-bca0-c501279c35b9 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:08:55.700574 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-9efb1245-8a87-4add-bca0-c501279c35b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:08:55.797692 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 838/1659] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:08:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 273 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:08:55.799995 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676d96c5-3ed5-4d15-b967-56bb70d81c4d None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:08:55.800292 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676d96c5-3ed5-4d15-b967-56bb70d81c4d None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:08:55.800572 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676d96c5-3ed5-4d15-b967-56bb70d81c4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:08:55.836323 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-676d96c5-3ed5-4d15-b967-56bb70d81c4d None None] MFA Rules not processed for user `bf55a00e3b084aa3b22c25a90e6a773c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:08:55.837409 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-676d96c5-3ed5-4d15-b967-56bb70d81c4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:08:55.950690 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efb1245-8a87-4add-bca0-c501279c35b9 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:08:55.983170 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2779394a-8d60-4d11-ae5e-57a350db8362 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:08:55.984083 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2779394a-8d60-4d11-ae5e-57a350db8362 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:08:55.984368 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2779394a-8d60-4d11-ae5e-57a350db8362 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:08:55.984648 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2779394a-8d60-4d11-ae5e-57a350db8362 admin admin] PATH_INFO: `/v3/users/b04b122515244068a9c83c62fca584a8` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:08:55.987571 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2779394a-8d60-4d11-ae5e-57a350db8362 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b04b122515244068a9c83c62fca584a8)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:08:55.990553 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2779394a-8d60-4d11-ae5e-57a350db8362 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:08:56.035969 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 839/1660] 10.222.0.46 () {58 vars in 1057 bytes} [Thu May 26 15:08:55 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:08:56.037314 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce02f18e-3a6d-47bc-9c83-773d1a244139 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:08:56.044052 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce02f18e-3a6d-47bc-9c83-773d1a244139 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:08:56.044433 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce02f18e-3a6d-47bc-9c83-773d1a244139 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:08:56.083793 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-ce02f18e-3a6d-47bc-9c83-773d1a244139 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:08:56.085936 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-ce02f18e-3a6d-47bc-9c83-773d1a244139 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:08:56.097626 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-2779394a-8d60-4d11-ae5e-57a350db8362 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b04b122515244068a9c83c62fca584a8'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:08:56.117815 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-2779394a-8d60-4d11-ae5e-57a350db8362 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b04b122515244068a9c83c62fca584a8'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:08:56.118112 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 840/1661] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:08:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:08:56.119064 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce02f18e-3a6d-47bc-9c83-773d1a244139 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:08:56.119854 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-2779394a-8d60-4d11-ae5e-57a350db8362 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b04b122515244068a9c83c62fca584a8'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:08:56.142602 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce02f18e-3a6d-47bc-9c83-773d1a244139 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:08:56.148464 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-2779394a-8d60-4d11-ae5e-57a350db8362 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b04b122515244068a9c83c62fca584a8'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:08:56.166388 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a17c7a-1d5b-42f5-b280-94f2dd7ba9cd service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf55a00e3b084aa3b22c25a90e6a773c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:08:56.168099 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a17c7a-1d5b-42f5-b280-94f2dd7ba9cd service placement] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:08:56.172601 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a17c7a-1d5b-42f5-b280-94f2dd7ba9cd service placement] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:08:56.172964 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a17c7a-1d5b-42f5-b280-94f2dd7ba9cd service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:08:56.174275 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-2779394a-8d60-4d11-ae5e-57a350db8362 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b04b122515244068a9c83c62fca584a8'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:08:56.177059 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-71a17c7a-1d5b-42f5-b280-94f2dd7ba9cd service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:08:56.191771 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 822/1662] 10.222.0.46 () {58 vars in 1223 bytes} [Thu May 26 15:08:55 2022] DELETE /identity/v3/users/b04b122515244068a9c83c62fca584a8 => generated 0 bytes in 494 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:08:56.192819 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2779394a-8d60-4d11-ae5e-57a350db8362 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:08:56.194291 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-2779394a-8d60-4d11-ae5e-57a350db8362 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:08:56.207403 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a17c7a-1d5b-42f5-b280-94f2dd7ba9cd service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:08:56.209023 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a17c7a-1d5b-42f5-b280-94f2dd7ba9cd service placement] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:08:56.242538 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 841/1663] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:08:56 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 15:08:56.377432 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:08:56.392841 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2779394a-8d60-4d11-ae5e-57a350db8362 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:08:56.406806 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fc3ea3-3709-4759-9df9-e4cfd5fed661 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:08:56.408148 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fc3ea3-3709-4759-9df9-e4cfd5fed661 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:08:56.408492 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fc3ea3-3709-4759-9df9-e4cfd5fed661 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:08:56.408716 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fc3ea3-3709-4759-9df9-e4cfd5fed661 admin admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:08:56.412895 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fc3ea3-3709-4759-9df9-e4cfd5fed661 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 15:08:56.413922 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fc3ea3-3709-4759-9df9-e4cfd5fed661 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:08:56.422086 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fc3ea3-3709-4759-9df9-e4cfd5fed661 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:08:56.451281 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 823/1664] 10.222.0.46 () {58 vars in 1152 bytes} [Thu May 26 15:08:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 15:08:56.456051 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a17c7a-1d5b-42f5-b280-94f2dd7ba9cd service placement] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:08:56.471724 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a17c7a-1d5b-42f5-b280-94f2dd7ba9cd service placement] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:08:56.489496 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93b5d7f-96ba-4155-a8c3-ac05f7066bd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:08:56.490172 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93b5d7f-96ba-4155-a8c3-ac05f7066bd0 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:08:56.490452 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93b5d7f-96ba-4155-a8c3-ac05f7066bd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:08:56.490724 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93b5d7f-96ba-4155-a8c3-ac05f7066bd0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:08:56.492572 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93b5d7f-96ba-4155-a8c3-ac05f7066bd0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:08:56.494621 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93b5d7f-96ba-4155-a8c3-ac05f7066bd0 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:08:56.565702 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 842/1665] 10.222.0.46 () {60 vars in 1152 bytes} [Thu May 26 15:08:56 2022] POST /identity/v3/projects => generated 363 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 15:08:56.569813 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fc3ea3-3709-4759-9df9-e4cfd5fed661 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:08:56.585062 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fc3ea3-3709-4759-9df9-e4cfd5fed661 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:08:56.759288 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cc14a4-ca6e-4bc7-9ab9-b18ae83de542 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:08:56.762651 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cc14a4-ca6e-4bc7-9ab9-b18ae83de542 admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:08:56.762651 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cc14a4-ca6e-4bc7-9ab9-b18ae83de542 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:08:56.762651 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cc14a4-ca6e-4bc7-9ab9-b18ae83de542 admin admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:08:56.765033 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cc14a4-ca6e-4bc7-9ab9-b18ae83de542 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:08:56.765806 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06cc14a4-ca6e-4bc7-9ab9-b18ae83de542 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:08:56.824854 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 824/1666] 10.222.0.46 () {60 vars in 1143 bytes} [Thu May 26 15:08:56 2022] POST /identity/v3/users => generated 282 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 15:08:56.830042 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93b5d7f-96ba-4155-a8c3-ac05f7066bd0 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:08:56.853015 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93b5d7f-96ba-4155-a8c3-ac05f7066bd0 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:08:56.866991 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed52836e-3ba8-4fcf-a5c3-44a5ae8ebd73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:08:56.867929 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed52836e-3ba8-4fcf-a5c3-44a5ae8ebd73 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:08:56.868021 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed52836e-3ba8-4fcf-a5c3-44a5ae8ebd73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:08:56.868265 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed52836e-3ba8-4fcf-a5c3-44a5ae8ebd73 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:08:56.869006 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed52836e-3ba8-4fcf-a5c3-44a5ae8ebd73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:08:56.878422 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed52836e-3ba8-4fcf-a5c3-44a5ae8ebd73 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:08:56.909494 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 843/1667] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 15:08:56 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 15:08:56.911074 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cc14a4-ca6e-4bc7-9ab9-b18ae83de542 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:08:56.928720 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cc14a4-ca6e-4bc7-9ab9-b18ae83de542 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:08:56.943726 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5f6293-81d3-473e-9825-b6eb8ab243b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:08:56.944773 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5f6293-81d3-473e-9825-b6eb8ab243b5 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:08:56.945070 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5f6293-81d3-473e-9825-b6eb8ab243b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:08:56.946569 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5f6293-81d3-473e-9825-b6eb8ab243b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:08:56.949332 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5f6293-81d3-473e-9825-b6eb8ab243b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:08:56.951254 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5f6293-81d3-473e-9825-b6eb8ab243b5 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:08:56.982174 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 825/1668] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 15:08:56 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 15:08:56.986839 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed52836e-3ba8-4fcf-a5c3-44a5ae8ebd73 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:08:57.026868 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed52836e-3ba8-4fcf-a5c3-44a5ae8ebd73 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:08:57.043450 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307274da-a698-4b91-aa65-12c7112b7fdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:08:57.044724 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307274da-a698-4b91-aa65-12c7112b7fdd admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:08:57.046003 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307274da-a698-4b91-aa65-12c7112b7fdd admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:08:57.046473 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307274da-a698-4b91-aa65-12c7112b7fdd admin admin] PATH_INFO: `/v3/projects/d8d41b2b380b48aa83e6a852a826615e/users/5bf28ae4216140e9a2bd5fc2d90ab48b/roles/d4088d1845a24fbd9bd8975ce0b1d3d0` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:08:57.057573 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=94,keystone:DELETE=4,keystone:INSERT=2 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:08:57.117657 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307274da-a698-4b91-aa65-12c7112b7fdd admin admin] RBAC: Authorizing `identity:create_grant(project_id=d8d41b2b380b48aa83e6a852a826615e, user_id=5bf28ae4216140e9a2bd5fc2d90ab48b, role_id=d4088d1845a24fbd9bd8975ce0b1d3d0)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:08:57.129018 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307274da-a698-4b91-aa65-12c7112b7fdd admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:08:57.165841 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 844/1669] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 15:08:56 2022] PUT /identity/v3/projects/d8d41b2b380b48aa83e6a852a826615e/users/5bf28ae4216140e9a2bd5fc2d90ab48b/roles/d4088d1845a24fbd9bd8975ce0b1d3d0 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:08:57.178481 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ce5bd5-9c0e-4d2a-bd65-b35234ff21a5 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:08:57.178885 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ce5bd5-9c0e-4d2a-bd65-b35234ff21a5 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:08:57.179064 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ce5bd5-9c0e-4d2a-bd65-b35234ff21a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:08:57.245081 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-43ce5bd5-9c0e-4d2a-bd65-b35234ff21a5 None None] MFA Rules not processed for user `5bf28ae4216140e9a2bd5fc2d90ab48b`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:08:57.247839 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-43ce5bd5-9c0e-4d2a-bd65-b35234ff21a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:08:57.512098 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 826/1670] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:08:57 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 335 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 15:08:57.518425 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307274da-a698-4b91-aa65-12c7112b7fdd admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:08:57.685444 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307274da-a698-4b91-aa65-12c7112b7fdd admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:08:57.701339 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4eebe1-c994-41d8-9e85-82ef2b2def74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:08:57.703793 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4eebe1-c994-41d8-9e85-82ef2b2def74 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:08:57.705843 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4eebe1-c994-41d8-9e85-82ef2b2def74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:08:57.706326 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4eebe1-c994-41d8-9e85-82ef2b2def74 admin admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:08:57.707277 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4eebe1-c994-41d8-9e85-82ef2b2def74 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:08:57.713730 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4eebe1-c994-41d8-9e85-82ef2b2def74 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:08:57.817417 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 845/1671] 10.222.0.46 () {60 vars in 1152 bytes} [Thu May 26 15:08:57 2022] POST /identity/v3/projects => generated 363 bytes in 297 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 15:08:57.818892 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ce5bd5-9c0e-4d2a-bd65-b35234ff21a5 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:08:57.839454 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ce5bd5-9c0e-4d2a-bd65-b35234ff21a5 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:08:57.921385 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5fd405-ba35-4667-b66e-9fce48714cc7 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:08:57.921700 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5fd405-ba35-4667-b66e-9fce48714cc7 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:08:57.921992 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5fd405-ba35-4667-b66e-9fce48714cc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:08:57.986584 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-ab5fd405-ba35-4667-b66e-9fce48714cc7 None None] MFA Rules not processed for user `fe5f60f71d5649bc8449be91da9127e9`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:08:57.987866 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-ab5fd405-ba35-4667-b66e-9fce48714cc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:08:58.010142 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:08:58.032249 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6500396c-519f-4220-b2f0-e65dd08caee3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:08:58.033792 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6500396c-519f-4220-b2f0-e65dd08caee3 admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:08:58.035598 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6500396c-519f-4220-b2f0-e65dd08caee3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:08:58.036009 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6500396c-519f-4220-b2f0-e65dd08caee3 admin admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:08:58.036812 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6500396c-519f-4220-b2f0-e65dd08caee3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:08:58.040825 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6500396c-519f-4220-b2f0-e65dd08caee3 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:08:58.099542 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 827/1672] 10.222.0.46 () {60 vars in 1143 bytes} [Thu May 26 15:08:57 2022] POST /identity/v3/users => generated 288 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 15:08:58.100529 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6500396c-519f-4220-b2f0-e65dd08caee3 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:08:58.132193 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6500396c-519f-4220-b2f0-e65dd08caee3 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:08:58.145326 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512d59a9-b3f6-4828-8ed4-5c8c756277a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:08:58.146167 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512d59a9-b3f6-4828-8ed4-5c8c756277a7 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:08:58.147599 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512d59a9-b3f6-4828-8ed4-5c8c756277a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:08:58.148311 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512d59a9-b3f6-4828-8ed4-5c8c756277a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:08:58.150745 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512d59a9-b3f6-4828-8ed4-5c8c756277a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:08:58.161757 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512d59a9-b3f6-4828-8ed4-5c8c756277a7 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:08:58.189551 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 828/1673] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 15:08:58 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 15:08:58.195382 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512d59a9-b3f6-4828-8ed4-5c8c756277a7 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:08:58.211426 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 846/1674] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:08:57 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 291 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:08:58.226614 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512d59a9-b3f6-4828-8ed4-5c8c756277a7 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:08:58.242787 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a597b2e-e5b6-4b34-9df2-4f4d7ac32877 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:08:58.243701 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a597b2e-e5b6-4b34-9df2-4f4d7ac32877 admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:08:58.244209 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a597b2e-e5b6-4b34-9df2-4f4d7ac32877 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:08:58.245859 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a597b2e-e5b6-4b34-9df2-4f4d7ac32877 admin admin] PATH_INFO: `/v3/projects/b2135fbd21c845c0ab71d02d5be2d3d4/users/b8213e6df38543db9bae0ca19d7f0aef/roles/a2e551773cbd4df084020676865b8222` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:08:58.278710 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a597b2e-e5b6-4b34-9df2-4f4d7ac32877 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b2135fbd21c845c0ab71d02d5be2d3d4, user_id=b8213e6df38543db9bae0ca19d7f0aef, role_id=a2e551773cbd4df084020676865b8222)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:08:58.283556 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a597b2e-e5b6-4b34-9df2-4f4d7ac32877 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:08:58.319658 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 829/1675] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 15:08:58 2022] PUT /identity/v3/projects/b2135fbd21c845c0ab71d02d5be2d3d4/users/b8213e6df38543db9bae0ca19d7f0aef/roles/a2e551773cbd4df084020676865b8222 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:08:58.323720 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5fd405-ba35-4667-b66e-9fce48714cc7 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:08:58.346498 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5fd405-ba35-4667-b66e-9fce48714cc7 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:08:58.441163 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a597b2e-e5b6-4b34-9df2-4f4d7ac32877 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:08:58.444255 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-5a597b2e-e5b6-4b34-9df2-4f4d7ac32877 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:08:58.554810 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:08:58.568323 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03e2e9c-954c-434d-863d-cd0816e1ac60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:08:58.569129 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03e2e9c-954c-434d-863d-cd0816e1ac60 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:08:58.569511 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03e2e9c-954c-434d-863d-cd0816e1ac60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:08:58.569801 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03e2e9c-954c-434d-863d-cd0816e1ac60 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:08:58.570607 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03e2e9c-954c-434d-863d-cd0816e1ac60 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:08:58.572276 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c03e2e9c-954c-434d-863d-cd0816e1ac60 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:08:58.593741 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 847/1676] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 15:08:58 2022] GET /identity/v3/roles => generated 1420 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 15:08:58.595022 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03e2e9c-954c-434d-863d-cd0816e1ac60 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:08:58.614344 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03e2e9c-954c-434d-863d-cd0816e1ac60 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:08:58.627353 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6fdd57-af02-4e38-be73-785d50b64f04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:08:58.630311 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6fdd57-af02-4e38-be73-785d50b64f04 admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:08:58.630659 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6fdd57-af02-4e38-be73-785d50b64f04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:08:58.630954 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6fdd57-af02-4e38-be73-785d50b64f04 admin admin] PATH_INFO: `/v3/projects/b2135fbd21c845c0ab71d02d5be2d3d4/users/b8213e6df38543db9bae0ca19d7f0aef/roles/a2e551773cbd4df084020676865b8222` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:08:58.633784 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6fdd57-af02-4e38-be73-785d50b64f04 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b2135fbd21c845c0ab71d02d5be2d3d4, user_id=b8213e6df38543db9bae0ca19d7f0aef, role_id=a2e551773cbd4df084020676865b8222)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:08:58.639137 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6fdd57-af02-4e38-be73-785d50b64f04 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:08:58.720154 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 848/1677] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 15:08:58 2022] PUT /identity/v3/projects/b2135fbd21c845c0ab71d02d5be2d3d4/users/b8213e6df38543db9bae0ca19d7f0aef/roles/a2e551773cbd4df084020676865b8222 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:08:58.722322 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac0d4c3-c970-4021-8fac-3ed5363ee0df None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:08:58.722821 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac0d4c3-c970-4021-8fac-3ed5363ee0df None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:08:58.723717 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac0d4c3-c970-4021-8fac-3ed5363ee0df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:08:58.760100 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-2ac0d4c3-c970-4021-8fac-3ed5363ee0df None None] MFA Rules not processed for user `b8213e6df38543db9bae0ca19d7f0aef`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:08:58.762144 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-2ac0d4c3-c970-4021-8fac-3ed5363ee0df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:08:58.918346 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 849/1678] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:08:58 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:08:58.931984 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac0d4c3-c970-4021-8fac-3ed5363ee0df None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:08:58.933377 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-2ac0d4c3-c970-4021-8fac-3ed5363ee0df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:08:58.991108 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a597b2e-e5b6-4b34-9df2-4f4d7ac32877 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:08:59.152835 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac0d4c3-c970-4021-8fac-3ed5363ee0df None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:08:59.168847 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38d58ab-3f5a-4414-b44e-4e05e7639464 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5b5b35614a547d783e6e77ddb8c1c03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:08:59.169956 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38d58ab-3f5a-4414-b44e-4e05e7639464 service cinder] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:08:59.170957 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38d58ab-3f5a-4414-b44e-4e05e7639464 service cinder] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:08:59.171377 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38d58ab-3f5a-4414-b44e-4e05e7639464 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:08:59.174835 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7fc487-6608-4393-b008-c183985dad37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:08:59.175800 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7fc487-6608-4393-b008-c183985dad37 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:08:59.180538 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7fc487-6608-4393-b008-c183985dad37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:08:59.180836 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7fc487-6608-4393-b008-c183985dad37 admin admin] PATH_INFO: `/v3/projects/18277aa7a5374fed932e84717059d8ac` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:08:59.189803 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7fc487-6608-4393-b008-c183985dad37 admin admin] RBAC: Authorizing `identity:delete_project(project_id=18277aa7a5374fed932e84717059d8ac)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:08:59.194141 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7fc487-6608-4393-b008-c183985dad37 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:08:59.198068 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38d58ab-3f5a-4414-b44e-4e05e7639464 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:08:59.199593 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38d58ab-3f5a-4414-b44e-4e05e7639464 service cinder] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:08:59.237054 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 850/1679] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:08:58 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 15:08:59.378319 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-0c7fc487-6608-4393-b008-c183985dad37 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '18277aa7a5374fed932e84717059d8ac', 'request_id': 'req-0c7fc487-6608-4393-b008-c183985dad37'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:08:59.386664 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-0c7fc487-6608-4393-b008-c183985dad37 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '18277aa7a5374fed932e84717059d8ac', 'request_id': 'req-0c7fc487-6608-4393-b008-c183985dad37'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:08:59.396711 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=90,keystone:INSERT=4,keystone:DELETE=6 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:08:59.398747 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-0c7fc487-6608-4393-b008-c183985dad37 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '18277aa7a5374fed932e84717059d8ac', 'request_id': 'req-0c7fc487-6608-4393-b008-c183985dad37'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:08:59.439597 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-0c7fc487-6608-4393-b008-c183985dad37 admin admin] The token cache is being invalidate because project 18277aa7a5374fed932e84717059d8ac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 15:08:59.440736 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-0c7fc487-6608-4393-b008-c183985dad37 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:08:59.441293 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38d58ab-3f5a-4414-b44e-4e05e7639464 service cinder] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:08:59.441784 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-0c7fc487-6608-4393-b008-c183985dad37 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:08:59.443889 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 830/1680] 10.222.0.46 () {58 vars in 1232 bytes} [Thu May 26 15:08:58 2022] DELETE /identity/v3/projects/18277aa7a5374fed932e84717059d8ac => generated 0 bytes in 1003 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:08:59.452225 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596fe756-9980-4adc-a2a2-1d08e853494c None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:08:59.455504 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38d58ab-3f5a-4414-b44e-4e05e7639464 service cinder] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:08:59.456850 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596fe756-9980-4adc-a2a2-1d08e853494c None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:08:59.457719 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596fe756-9980-4adc-a2a2-1d08e853494c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:08:59.540781 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-596fe756-9980-4adc-a2a2-1d08e853494c None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:08:59.542171 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-596fe756-9980-4adc-a2a2-1d08e853494c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:08:59.616571 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a474b7ab-9f09-4f29-8c24-77bd9fb06f4c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5b5b35614a547d783e6e77ddb8c1c03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:08:59.618104 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a474b7ab-9f09-4f29-8c24-77bd9fb06f4c service cinder] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:08:59.622982 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a474b7ab-9f09-4f29-8c24-77bd9fb06f4c service cinder] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:08:59.622982 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a474b7ab-9f09-4f29-8c24-77bd9fb06f4c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:08:59.629581 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-a474b7ab-9f09-4f29-8c24-77bd9fb06f4c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:08:59.672706 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:08:59.708431 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 831/1681] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:08:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:08:59.710036 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8daec9-78b0-4308-b18d-57c8d8fcfd35 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:08:59.711528 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8daec9-78b0-4308-b18d-57c8d8fcfd35 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:08:59.711528 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8daec9-78b0-4308-b18d-57c8d8fcfd35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:08:59.742181 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-bf8daec9-78b0-4308-b18d-57c8d8fcfd35 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:08:59.743510 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-bf8daec9-78b0-4308-b18d-57c8d8fcfd35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:08:59.760267 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 832/1682] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:08:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:08:59.760482 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8daec9-78b0-4308-b18d-57c8d8fcfd35 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:08:59.797489 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8daec9-78b0-4308-b18d-57c8d8fcfd35 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:08:59.803175 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a474b7ab-9f09-4f29-8c24-77bd9fb06f4c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:08:59.805748 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a474b7ab-9f09-4f29-8c24-77bd9fb06f4c service cinder] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:08:59.810744 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cae844-6763-404b-8ca5-1017922c2af2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:08:59.811688 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cae844-6763-404b-8ca5-1017922c2af2 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:08:59.812172 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cae844-6763-404b-8ca5-1017922c2af2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:08:59.812685 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cae844-6763-404b-8ca5-1017922c2af2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:08:59.813907 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9cae844-6763-404b-8ca5-1017922c2af2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 15:08:59.814358 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9cae844-6763-404b-8ca5-1017922c2af2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:08:59.828178 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9cae844-6763-404b-8ca5-1017922c2af2 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:08:59.838166 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 851/1683] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:08:59 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 15:08:59.839197 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a474b7ab-9f09-4f29-8c24-77bd9fb06f4c service cinder] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:08:59.844015 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-a474b7ab-9f09-4f29-8c24-77bd9fb06f4c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:08:59.856330 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 833/1684] 10.222.0.46 () {58 vars in 1152 bytes} [Thu May 26 15:08:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 15:08:59.859128 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cae844-6763-404b-8ca5-1017922c2af2 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:08:59.876012 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a474b7ab-9f09-4f29-8c24-77bd9fb06f4c service cinder] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:08:59.878458 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cae844-6763-404b-8ca5-1017922c2af2 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:08:59.884102 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393548ab-f92d-4438-9367-8c3ac68c6f7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:08:59.884832 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393548ab-f92d-4438-9367-8c3ac68c6f7f admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:08:59.885121 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393548ab-f92d-4438-9367-8c3ac68c6f7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:08:59.885640 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393548ab-f92d-4438-9367-8c3ac68c6f7f admin admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:08:59.886439 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393548ab-f92d-4438-9367-8c3ac68c6f7f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:08:59.888615 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393548ab-f92d-4438-9367-8c3ac68c6f7f admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:08:59.895185 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54532163-4b7a-4ab7-b5c2-7435d81295c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:08:59.896297 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54532163-4b7a-4ab7-b5c2-7435d81295c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:08:59.896695 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54532163-4b7a-4ab7-b5c2-7435d81295c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:08:59.897089 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54532163-4b7a-4ab7-b5c2-7435d81295c4 admin admin] PATH_INFO: `/v3/users/eb0888d2b5024d558d15fe47e2d34ccb` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:08:59.912672 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54532163-4b7a-4ab7-b5c2-7435d81295c4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eb0888d2b5024d558d15fe47e2d34ccb)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:08:59.917417 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54532163-4b7a-4ab7-b5c2-7435d81295c4 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:08:59.937495 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 834/1685] 10.222.0.46 () {60 vars in 1152 bytes} [Thu May 26 15:08:59 2022] POST /identity/v3/projects => generated 381 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 15:08:59.943022 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393548ab-f92d-4438-9367-8c3ac68c6f7f admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:00.016853 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-54532163-4b7a-4ab7-b5c2-7435d81295c4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eb0888d2b5024d558d15fe47e2d34ccb'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:00.030611 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-54532163-4b7a-4ab7-b5c2-7435d81295c4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eb0888d2b5024d558d15fe47e2d34ccb'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:00.032634 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-54532163-4b7a-4ab7-b5c2-7435d81295c4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eb0888d2b5024d558d15fe47e2d34ccb'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:00.061944 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-54532163-4b7a-4ab7-b5c2-7435d81295c4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eb0888d2b5024d558d15fe47e2d34ccb'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:00.078212 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-54532163-4b7a-4ab7-b5c2-7435d81295c4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eb0888d2b5024d558d15fe47e2d34ccb'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:00.088405 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 852/1686] 10.222.0.46 () {58 vars in 1223 bytes} [Thu May 26 15:08:59 2022] DELETE /identity/v3/users/eb0888d2b5024d558d15fe47e2d34ccb => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:09:00.093669 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a1ce83-3069-4ebc-a61c-6b74158211f4 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:00.094278 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a1ce83-3069-4ebc-a61c-6b74158211f4 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:00.094770 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a1ce83-3069-4ebc-a61c-6b74158211f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:00.130623 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-b7a1ce83-3069-4ebc-a61c-6b74158211f4 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:09:00.132275 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-b7a1ce83-3069-4ebc-a61c-6b74158211f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:09:00.148054 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393548ab-f92d-4438-9367-8c3ac68c6f7f admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:00.281524 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=97,keystone:INSERT=2,keystone:DELETE=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:09:00.296616 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48acce15-7886-4ffd-822c-e7e66cb705a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:00.297311 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48acce15-7886-4ffd-822c-e7e66cb705a3 admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:00.297650 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48acce15-7886-4ffd-822c-e7e66cb705a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:00.298507 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48acce15-7886-4ffd-822c-e7e66cb705a3 admin admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:00.303290 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48acce15-7886-4ffd-822c-e7e66cb705a3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:00.310073 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48acce15-7886-4ffd-822c-e7e66cb705a3 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:00.336332 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 853/1687] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:09:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 15:09:00.387743 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a1ce83-3069-4ebc-a61c-6b74158211f4 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:00.388145 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 835/1688] 10.222.0.46 () {60 vars in 1143 bytes} [Thu May 26 15:08:59 2022] POST /identity/v3/users => generated 291 bytes in 445 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 15:09:00.390146 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-b7a1ce83-3069-4ebc-a61c-6b74158211f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:09:00.413727 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a1ce83-3069-4ebc-a61c-6b74158211f4 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:00.416576 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48acce15-7886-4ffd-822c-e7e66cb705a3 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:00.419114 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-48acce15-7886-4ffd-822c-e7e66cb705a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:09:00.429749 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58bdbe5-20cc-43da-a242-8eba18de0ec2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:00.430795 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58bdbe5-20cc-43da-a242-8eba18de0ec2 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:00.432693 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58bdbe5-20cc-43da-a242-8eba18de0ec2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:00.433253 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58bdbe5-20cc-43da-a242-8eba18de0ec2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:00.436594 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58bdbe5-20cc-43da-a242-8eba18de0ec2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:00.439135 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58bdbe5-20cc-43da-a242-8eba18de0ec2 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:00.453345 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 854/1689] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 15:09:00 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 15:09:00.470918 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58bdbe5-20cc-43da-a242-8eba18de0ec2 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:00.494144 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58bdbe5-20cc-43da-a242-8eba18de0ec2 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:00.501394 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d2aeb9-5723-4cde-a618-7620209dfd1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:00.502426 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d2aeb9-5723-4cde-a618-7620209dfd1e admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:00.502920 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d2aeb9-5723-4cde-a618-7620209dfd1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:00.503406 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d2aeb9-5723-4cde-a618-7620209dfd1e admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:00.504408 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d2aeb9-5723-4cde-a618-7620209dfd1e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:00.506531 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d2aeb9-5723-4cde-a618-7620209dfd1e admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:00.520770 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 855/1690] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 15:09:00 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 15:09:00.527450 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d2aeb9-5723-4cde-a618-7620209dfd1e admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:00.542530 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d2aeb9-5723-4cde-a618-7620209dfd1e admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:00.556687 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa99ea35-9977-46d1-9be4-59649cf6abdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:00.557828 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa99ea35-9977-46d1-9be4-59649cf6abdc admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:00.558225 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa99ea35-9977-46d1-9be4-59649cf6abdc admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:00.558616 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa99ea35-9977-46d1-9be4-59649cf6abdc admin admin] PATH_INFO: `/v3/projects/5792798bb80e49e895262713d5368988/users/4214e30903b543ada0a7edad0cf1e8ef/roles/d4088d1845a24fbd9bd8975ce0b1d3d0` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:00.578476 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:09:00.594443 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48acce15-7886-4ffd-822c-e7e66cb705a3 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:00.600690 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c521e2b0-bc03-40a2-958e-066ff5b7520e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0156a2f6f3794f479e76b85bdbbe5c26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:00.601649 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c521e2b0-bc03-40a2-958e-066ff5b7520e service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:00.602081 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c521e2b0-bc03-40a2-958e-066ff5b7520e service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:00.602739 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c521e2b0-bc03-40a2-958e-066ff5b7520e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:00.613661 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa99ea35-9977-46d1-9be4-59649cf6abdc admin admin] RBAC: Authorizing `identity:create_grant(project_id=5792798bb80e49e895262713d5368988, user_id=4214e30903b543ada0a7edad0cf1e8ef, role_id=d4088d1845a24fbd9bd8975ce0b1d3d0)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:00.621533 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa99ea35-9977-46d1-9be4-59649cf6abdc admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:00.632488 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c521e2b0-bc03-40a2-958e-066ff5b7520e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:00.634341 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c521e2b0-bc03-40a2-958e-066ff5b7520e service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:00.658431 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 856/1691] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 15:09:00 2022] PUT /identity/v3/projects/5792798bb80e49e895262713d5368988/users/4214e30903b543ada0a7edad0cf1e8ef/roles/d4088d1845a24fbd9bd8975ce0b1d3d0 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:09:00.663095 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cac02fe-a5a0-47b8-9811-78e13ef42054 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:00.664139 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cac02fe-a5a0-47b8-9811-78e13ef42054 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:00.665235 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cac02fe-a5a0-47b8-9811-78e13ef42054 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:00.674618 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 836/1692] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:09:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 259 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 15:09:00.705930 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96288b70-aa68-4f75-ae3a-e2ea01d281b7 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:00.707277 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96288b70-aa68-4f75-ae3a-e2ea01d281b7 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:00.707992 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96288b70-aa68-4f75-ae3a-e2ea01d281b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:00.731014 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-96288b70-aa68-4f75-ae3a-e2ea01d281b7 None None] MFA Rules not processed for user `8028c520d93449a6a087c281280ee13d`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:09:00.733999 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-96288b70-aa68-4f75-ae3a-e2ea01d281b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:09:00.738857 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-8cac02fe-a5a0-47b8-9811-78e13ef42054 None None] MFA Rules not processed for user `4214e30903b543ada0a7edad0cf1e8ef`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:09:00.741527 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-8cac02fe-a5a0-47b8-9811-78e13ef42054 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:09:00.860812 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 837/1693] 10.222.0.46 () {58 vars in 1009 bytes} [Thu May 26 15:09:00 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 15:09:00.902649 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 857/1694] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:09:00 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:09:00.908232 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96288b70-aa68-4f75-ae3a-e2ea01d281b7 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:01.008815 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96288b70-aa68-4f75-ae3a-e2ea01d281b7 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:01.019954 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48881df9-f3de-4451-bf6d-d6e66b46153e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:01.021400 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48881df9-f3de-4451-bf6d-d6e66b46153e admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:01.023265 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48881df9-f3de-4451-bf6d-d6e66b46153e admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:01.023709 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48881df9-f3de-4451-bf6d-d6e66b46153e admin admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:01.026172 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48881df9-f3de-4451-bf6d-d6e66b46153e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:01.031888 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48881df9-f3de-4451-bf6d-d6e66b46153e admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:01.110884 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 838/1695] 10.222.0.46 () {60 vars in 1152 bytes} [Thu May 26 15:09:00 2022] POST /identity/v3/projects => generated 381 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 15:09:01.114562 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cac02fe-a5a0-47b8-9811-78e13ef42054 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:01.121347 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48881df9-f3de-4451-bf6d-d6e66b46153e admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:01.128331 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-48881df9-f3de-4451-bf6d-d6e66b46153e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:09:01.144941 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cac02fe-a5a0-47b8-9811-78e13ef42054 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:01.234738 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:09:01.362698 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48881df9-f3de-4451-bf6d-d6e66b46153e admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:01.382145 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf682e84-4d98-49f8-8194-6a25bd5806bc service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4bea272624b45d8ac2ad27bd07df885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:01.383211 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf682e84-4d98-49f8-8194-6a25bd5806bc service swift] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:01.383707 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf682e84-4d98-49f8-8194-6a25bd5806bc service swift] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:01.384263 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf682e84-4d98-49f8-8194-6a25bd5806bc service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:01.389795 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ec2ee3-7535-4a64-bb90-3bbb761f8429 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:01.390734 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ec2ee3-7535-4a64-bb90-3bbb761f8429 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:01.391552 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ec2ee3-7535-4a64-bb90-3bbb761f8429 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:01.392109 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ec2ee3-7535-4a64-bb90-3bbb761f8429 admin admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:01.392623 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ec2ee3-7535-4a64-bb90-3bbb761f8429 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:01.399086 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ec2ee3-7535-4a64-bb90-3bbb761f8429 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:01.473499 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 858/1696] 10.222.0.46 () {60 vars in 1143 bytes} [Thu May 26 15:09:01 2022] POST /identity/v3/users => generated 297 bytes in 360 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 15:09:01.475730 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ec2ee3-7535-4a64-bb90-3bbb761f8429 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:01.509011 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ec2ee3-7535-4a64-bb90-3bbb761f8429 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:01.518227 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf682e84-4d98-49f8-8194-6a25bd5806bc service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:01.520471 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf682e84-4d98-49f8-8194-6a25bd5806bc service swift] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:01.524858 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66414f69-02e7-474f-af02-815c01a79bf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:01.527817 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66414f69-02e7-474f-af02-815c01a79bf4 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:01.528118 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66414f69-02e7-474f-af02-815c01a79bf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:01.528415 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66414f69-02e7-474f-af02-815c01a79bf4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:01.529193 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66414f69-02e7-474f-af02-815c01a79bf4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:01.535512 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66414f69-02e7-474f-af02-815c01a79bf4 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:01.546863 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 839/1697] 10.222.0.46 () {60 vars in 1384 bytes} [Thu May 26 15:09:01 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 26 15:09:01.621242 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf682e84-4d98-49f8-8194-6a25bd5806bc service swift] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:01.629723 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 859/1698] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 15:09:01 2022] GET /identity/v3/roles => generated 1420 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 15:09:01.660256 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf682e84-4d98-49f8-8194-6a25bd5806bc service swift] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:01.688051 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe2fc51-05d1-4521-b6ec-0ad58d666ff2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:01.693297 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe2fc51-05d1-4521-b6ec-0ad58d666ff2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:01.694775 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe2fc51-05d1-4521-b6ec-0ad58d666ff2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:01.696242 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe2fc51-05d1-4521-b6ec-0ad58d666ff2 admin admin] PATH_INFO: `/v3/projects/95ad0ab555bd4392926e926e53458a2f/users/7145c0ae0f3e4965907bcf282bdded82/roles/a2e551773cbd4df084020676865b8222` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:01.765415 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe2fc51-05d1-4521-b6ec-0ad58d666ff2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=95ad0ab555bd4392926e926e53458a2f, user_id=7145c0ae0f3e4965907bcf282bdded82, role_id=a2e551773cbd4df084020676865b8222)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:01.776495 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe2fc51-05d1-4521-b6ec-0ad58d666ff2 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:01.835494 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 840/1699] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 15:09:01 2022] PUT /identity/v3/projects/95ad0ab555bd4392926e926e53458a2f/users/7145c0ae0f3e4965907bcf282bdded82/roles/a2e551773cbd4df084020676865b8222 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:09:01.837006 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66414f69-02e7-474f-af02-815c01a79bf4 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:01.860846 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66414f69-02e7-474f-af02-815c01a79bf4 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:02.091719 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:09:02.131228 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0773131b-1f8e-4b6f-80d0-2e5739e308e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:02.133406 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0773131b-1f8e-4b6f-80d0-2e5739e308e4 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:02.133962 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0773131b-1f8e-4b6f-80d0-2e5739e308e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:02.144933 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0773131b-1f8e-4b6f-80d0-2e5739e308e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:02.147340 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0773131b-1f8e-4b6f-80d0-2e5739e308e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:02.149065 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0773131b-1f8e-4b6f-80d0-2e5739e308e4 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:02.224874 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 860/1700] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 15:09:01 2022] GET /identity/v3/roles => generated 1420 bytes in 389 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 15:09:02.228745 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe2fc51-05d1-4521-b6ec-0ad58d666ff2 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:02.248593 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe2fc51-05d1-4521-b6ec-0ad58d666ff2 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:02.289832 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5244715-312a-4b0d-82b9-216c4c4e5a26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:02.290885 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5244715-312a-4b0d-82b9-216c4c4e5a26 admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:02.313349 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5244715-312a-4b0d-82b9-216c4c4e5a26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:02.327150 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5244715-312a-4b0d-82b9-216c4c4e5a26 admin admin] PATH_INFO: `/v3/projects/95ad0ab555bd4392926e926e53458a2f/users/7145c0ae0f3e4965907bcf282bdded82/roles/a2e551773cbd4df084020676865b8222` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:02.329000 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5244715-312a-4b0d-82b9-216c4c4e5a26 admin admin] RBAC: Authorizing `identity:create_grant(project_id=95ad0ab555bd4392926e926e53458a2f, user_id=7145c0ae0f3e4965907bcf282bdded82, role_id=a2e551773cbd4df084020676865b8222)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:02.335052 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5244715-312a-4b0d-82b9-216c4c4e5a26 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:02.417855 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 841/1701] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 15:09:02 2022] PUT /identity/v3/projects/95ad0ab555bd4392926e926e53458a2f/users/7145c0ae0f3e4965907bcf282bdded82/roles/a2e551773cbd4df084020676865b8222 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:09:02.419926 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23859e11-6902-4bdb-9227-d79433d2ad08 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:02.420393 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23859e11-6902-4bdb-9227-d79433d2ad08 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:02.420559 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23859e11-6902-4bdb-9227-d79433d2ad08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:02.589141 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-23859e11-6902-4bdb-9227-d79433d2ad08 None None] MFA Rules not processed for user `7145c0ae0f3e4965907bcf282bdded82`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:09:02.590626 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-23859e11-6902-4bdb-9227-d79433d2ad08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:09:02.841106 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1eb832-0fa5-4374-b81e-2741a10f38aa None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:02.842313 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1eb832-0fa5-4374-b81e-2741a10f38aa None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:02.842758 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1eb832-0fa5-4374-b81e-2741a10f38aa None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:02.844821 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 842/1702] 10.222.0.46 () {54 vars in 921 bytes} [Thu May 26 15:09:02 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 15:09:02.977167 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 861/1703] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:09:02 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 558 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:09:02.992079 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1eb832-0fa5-4374-b81e-2741a10f38aa None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:02.993992 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-cc1eb832-0fa5-4374-b81e-2741a10f38aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:09:03.049030 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d29c3e7-2fbf-470b-b5e4-aa97fc3adb10 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:03.049273 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d29c3e7-2fbf-470b-b5e4-aa97fc3adb10 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:03.049913 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d29c3e7-2fbf-470b-b5e4-aa97fc3adb10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:03.099944 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-9d29c3e7-2fbf-470b-b5e4-aa97fc3adb10 None None] MFA Rules not processed for user `bf55a00e3b084aa3b22c25a90e6a773c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:09:03.102141 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-9d29c3e7-2fbf-470b-b5e4-aa97fc3adb10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:09:03.256293 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1eb832-0fa5-4374-b81e-2741a10f38aa None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:03.277945 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd00053-dadc-4f3a-827c-2077280ca2dd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5b5b35614a547d783e6e77ddb8c1c03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:03.281835 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd00053-dadc-4f3a-827c-2077280ca2dd service cinder] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:03.303207 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd00053-dadc-4f3a-827c-2077280ca2dd service cinder] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:03.303916 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd00053-dadc-4f3a-827c-2077280ca2dd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:03.323424 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd00053-dadc-4f3a-827c-2077280ca2dd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:03.325182 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd00053-dadc-4f3a-827c-2077280ca2dd service cinder] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:03.362823 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 843/1704] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:09:02 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 15:09:03.391023 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 862/1705] 10.222.0.46 () {58 vars in 1018 bytes} [Thu May 26 15:09:03 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 343 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:09:03.631211 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd00053-dadc-4f3a-827c-2077280ca2dd service cinder] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:03.648874 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd00053-dadc-4f3a-827c-2077280ca2dd service cinder] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:03.657895 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf76764-3638-4e91-a456-bb62347905c8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5b5b35614a547d783e6e77ddb8c1c03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:03.658762 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf76764-3638-4e91-a456-bb62347905c8 service cinder] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:03.659048 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf76764-3638-4e91-a456-bb62347905c8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:03.659332 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf76764-3638-4e91-a456-bb62347905c8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:03.741951 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:09:03.910610 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf76764-3638-4e91-a456-bb62347905c8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:03.914919 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf76764-3638-4e91-a456-bb62347905c8 service cinder] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:03.959632 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 844/1706] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:09:03 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 15:09:03.971403 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d29c3e7-2fbf-470b-b5e4-aa97fc3adb10 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:03.974491 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-9d29c3e7-2fbf-470b-b5e4-aa97fc3adb10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:09:04.006920 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d29c3e7-2fbf-470b-b5e4-aa97fc3adb10 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:04.032538 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7241f00b-e9a6-48a1-9a0d-0dd875f2da99 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf55a00e3b084aa3b22c25a90e6a773c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:04.035457 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7241f00b-e9a6-48a1-9a0d-0dd875f2da99 service placement] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:04.035744 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7241f00b-e9a6-48a1-9a0d-0dd875f2da99 service placement] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:04.035978 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7241f00b-e9a6-48a1-9a0d-0dd875f2da99 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:04.060425 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7241f00b-e9a6-48a1-9a0d-0dd875f2da99 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:04.069044 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7241f00b-e9a6-48a1-9a0d-0dd875f2da99 service placement] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:04.119252 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 863/1707] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:09:03 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 15:09:04.897730 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf76764-3638-4e91-a456-bb62347905c8 service cinder] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:04.908504 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-8cf76764-3638-4e91-a456-bb62347905c8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:09:05.274714 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf76764-3638-4e91-a456-bb62347905c8 service cinder] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:05.287212 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996abde5-3420-44ba-829f-886925b966c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:05.290605 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996abde5-3420-44ba-829f-886925b966c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:05.294321 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996abde5-3420-44ba-829f-886925b966c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:05.296790 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996abde5-3420-44ba-829f-886925b966c5 admin admin] PATH_INFO: `/v3/projects/595a3d59c45f452d935dcec7a8938828` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:05.322387 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996abde5-3420-44ba-829f-886925b966c5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=595a3d59c45f452d935dcec7a8938828)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:05.337362 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996abde5-3420-44ba-829f-886925b966c5 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:05.500563 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-996abde5-3420-44ba-829f-886925b966c5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '595a3d59c45f452d935dcec7a8938828', 'request_id': 'req-996abde5-3420-44ba-829f-886925b966c5'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:05.538604 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-996abde5-3420-44ba-829f-886925b966c5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '595a3d59c45f452d935dcec7a8938828', 'request_id': 'req-996abde5-3420-44ba-829f-886925b966c5'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:05.577798 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-996abde5-3420-44ba-829f-886925b966c5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '595a3d59c45f452d935dcec7a8938828', 'request_id': 'req-996abde5-3420-44ba-829f-886925b966c5'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:05.629913 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-996abde5-3420-44ba-829f-886925b966c5 admin admin] The token cache is being invalidate because project 595a3d59c45f452d935dcec7a8938828 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 15:09:05.633784 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-996abde5-3420-44ba-829f-886925b966c5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:05.636159 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-996abde5-3420-44ba-829f-886925b966c5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:05.647107 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 845/1708] 10.222.0.46 () {58 vars in 1232 bytes} [Thu May 26 15:09:04 2022] DELETE /identity/v3/projects/595a3d59c45f452d935dcec7a8938828 => generated 0 bytes in 750 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:09:05.653572 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493a0064-5582-44b6-8947-3ec68cd7a7e7 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:05.655192 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493a0064-5582-44b6-8947-3ec68cd7a7e7 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:05.655747 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493a0064-5582-44b6-8947-3ec68cd7a7e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:05.702040 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-493a0064-5582-44b6-8947-3ec68cd7a7e7 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:09:05.707590 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-493a0064-5582-44b6-8947-3ec68cd7a7e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:09:05.872373 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:09:05.899343 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 864/1709] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:09:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:09:05.904062 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa04800-e7b2-43df-a74f-95901a9c9330 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:05.905923 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa04800-e7b2-43df-a74f-95901a9c9330 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:05.906439 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa04800-e7b2-43df-a74f-95901a9c9330 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:05.942559 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-efa04800-e7b2-43df-a74f-95901a9c9330 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:09:05.944326 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-efa04800-e7b2-43df-a74f-95901a9c9330 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:09:05.956992 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 846/1710] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:09:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:09:05.961260 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493a0064-5582-44b6-8947-3ec68cd7a7e7 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:05.987152 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493a0064-5582-44b6-8947-3ec68cd7a7e7 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:06.004899 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6a1a6a-f58a-4141-9c91-9068caf9724e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:06.008286 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6a1a6a-f58a-4141-9c91-9068caf9724e admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:06.010120 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6a1a6a-f58a-4141-9c91-9068caf9724e admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:06.010682 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6a1a6a-f58a-4141-9c91-9068caf9724e admin admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:06.013110 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6a1a6a-f58a-4141-9c91-9068caf9724e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 15:09:06.016626 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6a1a6a-f58a-4141-9c91-9068caf9724e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:06.024887 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c6a1a6a-f58a-4141-9c91-9068caf9724e admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:06.078305 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 865/1711] 10.222.0.46 () {58 vars in 1152 bytes} [Thu May 26 15:09:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 15:09:06.081508 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa04800-e7b2-43df-a74f-95901a9c9330 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:06.105267 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa04800-e7b2-43df-a74f-95901a9c9330 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:06.132845 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17025dfe-a6cb-4a39-97c1-03a1ea1be9eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:06.134213 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17025dfe-a6cb-4a39-97c1-03a1ea1be9eb admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:06.136665 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17025dfe-a6cb-4a39-97c1-03a1ea1be9eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:06.139612 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17025dfe-a6cb-4a39-97c1-03a1ea1be9eb admin admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:06.143443 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17025dfe-a6cb-4a39-97c1-03a1ea1be9eb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:06.154840 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17025dfe-a6cb-4a39-97c1-03a1ea1be9eb admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:06.309125 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 847/1712] 10.222.0.46 () {60 vars in 1152 bytes} [Thu May 26 15:09:06 2022] POST /identity/v3/projects => generated 363 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 15:09:06.315078 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6a1a6a-f58a-4141-9c91-9068caf9724e admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:06.370563 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6a1a6a-f58a-4141-9c91-9068caf9724e admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:06.600377 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b71028e-5aa7-4718-81e0-0028c5e0f8d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:06.601439 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b71028e-5aa7-4718-81e0-0028c5e0f8d5 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:06.601771 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b71028e-5aa7-4718-81e0-0028c5e0f8d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:06.602059 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b71028e-5aa7-4718-81e0-0028c5e0f8d5 admin admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:06.602873 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b71028e-5aa7-4718-81e0-0028c5e0f8d5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:06.609268 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b71028e-5aa7-4718-81e0-0028c5e0f8d5 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:06.635981 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 866/1713] 10.222.0.46 () {60 vars in 1143 bytes} [Thu May 26 15:09:06 2022] POST /identity/v3/users => generated 282 bytes in 321 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 15:09:06.640355 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17025dfe-a6cb-4a39-97c1-03a1ea1be9eb admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:06.666645 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17025dfe-a6cb-4a39-97c1-03a1ea1be9eb admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:06.690435 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4730bd-6a02-4115-9f79-56a4dde22a91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:06.691636 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4730bd-6a02-4115-9f79-56a4dde22a91 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:06.692012 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4730bd-6a02-4115-9f79-56a4dde22a91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:06.692552 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4730bd-6a02-4115-9f79-56a4dde22a91 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:06.693381 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4730bd-6a02-4115-9f79-56a4dde22a91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:06.695146 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4730bd-6a02-4115-9f79-56a4dde22a91 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:06.726589 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 848/1714] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 15:09:06 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 15:09:06.730681 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b71028e-5aa7-4718-81e0-0028c5e0f8d5 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:06.750696 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b71028e-5aa7-4718-81e0-0028c5e0f8d5 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:06.769231 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f57f3dd-50f8-41b7-8efc-14272bd96d88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:06.770552 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f57f3dd-50f8-41b7-8efc-14272bd96d88 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:06.771205 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f57f3dd-50f8-41b7-8efc-14272bd96d88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:06.771935 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f57f3dd-50f8-41b7-8efc-14272bd96d88 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:06.773466 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f57f3dd-50f8-41b7-8efc-14272bd96d88 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:06.780522 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f57f3dd-50f8-41b7-8efc-14272bd96d88 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:06.819684 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4730bd-6a02-4115-9f79-56a4dde22a91 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:06.827887 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 867/1715] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 15:09:06 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 15:09:06.836627 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4730bd-6a02-4115-9f79-56a4dde22a91 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:06.852724 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3e50e3-19d1-4f53-8854-157d799299ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:06.853814 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3e50e3-19d1-4f53-8854-157d799299ce admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:06.854206 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3e50e3-19d1-4f53-8854-157d799299ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:06.855746 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3e50e3-19d1-4f53-8854-157d799299ce admin admin] PATH_INFO: `/v3/projects/ed634dcb373941e2b612f1402f5aedb2/users/8e3e2483a9f94a759da1bb8fab329987/roles/d4088d1845a24fbd9bd8975ce0b1d3d0` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:06.898881 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:09:06.914448 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3e50e3-19d1-4f53-8854-157d799299ce admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed634dcb373941e2b612f1402f5aedb2, user_id=8e3e2483a9f94a759da1bb8fab329987, role_id=d4088d1845a24fbd9bd8975ce0b1d3d0)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:06.918839 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3e50e3-19d1-4f53-8854-157d799299ce admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:06.947368 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 849/1716] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 15:09:06 2022] PUT /identity/v3/projects/ed634dcb373941e2b612f1402f5aedb2/users/8e3e2483a9f94a759da1bb8fab329987/roles/d4088d1845a24fbd9bd8975ce0b1d3d0 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 15:09:06.958507 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880eaab0-4494-4f24-a328-720c999beae0 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:06.958825 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880eaab0-4494-4f24-a328-720c999beae0 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:06.959089 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880eaab0-4494-4f24-a328-720c999beae0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:07.042613 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-880eaab0-4494-4f24-a328-720c999beae0 None None] MFA Rules not processed for user `8e3e2483a9f94a759da1bb8fab329987`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:09:07.043953 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-880eaab0-4494-4f24-a328-720c999beae0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:09:07.295955 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 868/1717] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:09:06 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 338 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:09:07.329127 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3e50e3-19d1-4f53-8854-157d799299ce admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:07.331957 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-fa3e50e3-19d1-4f53-8854-157d799299ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:09:07.381227 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880eaab0-4494-4f24-a328-720c999beae0 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:07.383940 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-880eaab0-4494-4f24-a328-720c999beae0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:09:07.575593 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3e50e3-19d1-4f53-8854-157d799299ce admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:07.589307 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6381a42-fd27-4cf5-8df7-fe913b665a73 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5b5b35614a547d783e6e77ddb8c1c03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:07.589883 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6381a42-fd27-4cf5-8df7-fe913b665a73 service cinder] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:07.590163 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6381a42-fd27-4cf5-8df7-fe913b665a73 service cinder] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:07.590443 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6381a42-fd27-4cf5-8df7-fe913b665a73 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:07.608320 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:09:07.618286 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6381a42-fd27-4cf5-8df7-fe913b665a73 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:07.622470 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6381a42-fd27-4cf5-8df7-fe913b665a73 service cinder] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:07.629867 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880eaab0-4494-4f24-a328-720c999beae0 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:07.643407 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea7964d-87b0-4316-a693-ea752c7134c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:07.644336 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea7964d-87b0-4316-a693-ea752c7134c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:07.644694 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea7964d-87b0-4316-a693-ea752c7134c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:07.644898 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea7964d-87b0-4316-a693-ea752c7134c9 admin admin] PATH_INFO: `/v3/users/8242513dc8c84bada9d2b647064cf06c` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:07.651637 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 850/1718] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:09:07 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 15:09:07.652532 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea7964d-87b0-4316-a693-ea752c7134c9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8242513dc8c84bada9d2b647064cf06c)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:07.654733 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea7964d-87b0-4316-a693-ea752c7134c9 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:07.800918 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-bea7964d-87b0-4316-a693-ea752c7134c9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8242513dc8c84bada9d2b647064cf06c'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:07.823737 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-bea7964d-87b0-4316-a693-ea752c7134c9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8242513dc8c84bada9d2b647064cf06c'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:07.825147 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-bea7964d-87b0-4316-a693-ea752c7134c9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8242513dc8c84bada9d2b647064cf06c'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:07.845335 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-bea7964d-87b0-4316-a693-ea752c7134c9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8242513dc8c84bada9d2b647064cf06c'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:07.856245 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-bea7964d-87b0-4316-a693-ea752c7134c9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8242513dc8c84bada9d2b647064cf06c'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:07.866163 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 869/1719] 10.222.0.46 () {58 vars in 1223 bytes} [Thu May 26 15:09:07 2022] DELETE /identity/v3/users/8242513dc8c84bada9d2b647064cf06c => generated 0 bytes in 486 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:09:08.443733 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6381a42-fd27-4cf5-8df7-fe913b665a73 service cinder] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:08.492969 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6381a42-fd27-4cf5-8df7-fe913b665a73 service cinder] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:08.644795 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb271989-627e-461b-ae04-1ac43abd4e1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:08.646052 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb271989-627e-461b-ae04-1ac43abd4e1c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:08.646425 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb271989-627e-461b-ae04-1ac43abd4e1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:08.648326 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb271989-627e-461b-ae04-1ac43abd4e1c admin admin] PATH_INFO: `/v3/projects/68969c3e05e9433dbac7ff643427c5f4` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:08.668974 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cb627c-ec58-4097-b0a6-ad14c6bfc00e None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:08.669184 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cb627c-ec58-4097-b0a6-ad14c6bfc00e None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:08.669517 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cb627c-ec58-4097-b0a6-ad14c6bfc00e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:08.670826 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb271989-627e-461b-ae04-1ac43abd4e1c admin admin] RBAC: Authorizing `identity:delete_project(project_id=68969c3e05e9433dbac7ff643427c5f4)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:08.672688 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb271989-627e-461b-ae04-1ac43abd4e1c admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:08.744963 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-c6cb627c-ec58-4097-b0a6-ad14c6bfc00e None None] MFA Rules not processed for user `64aeada36ebe460e9969f1f048651e6d`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:09:08.747766 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-c6cb627c-ec58-4097-b0a6-ad14c6bfc00e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:09:08.764576 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-bb271989-627e-461b-ae04-1ac43abd4e1c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '68969c3e05e9433dbac7ff643427c5f4', 'request_id': 'req-bb271989-627e-461b-ae04-1ac43abd4e1c'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:08.772332 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-bb271989-627e-461b-ae04-1ac43abd4e1c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '68969c3e05e9433dbac7ff643427c5f4', 'request_id': 'req-bb271989-627e-461b-ae04-1ac43abd4e1c'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:08.780440 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-bb271989-627e-461b-ae04-1ac43abd4e1c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '68969c3e05e9433dbac7ff643427c5f4', 'request_id': 'req-bb271989-627e-461b-ae04-1ac43abd4e1c'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:08.824544 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-bb271989-627e-461b-ae04-1ac43abd4e1c admin admin] The token cache is being invalidate because project 68969c3e05e9433dbac7ff643427c5f4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 15:09:08.824925 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-bb271989-627e-461b-ae04-1ac43abd4e1c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:08.825591 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-bb271989-627e-461b-ae04-1ac43abd4e1c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:08.831156 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 851/1720] 10.222.0.46 () {58 vars in 1232 bytes} [Thu May 26 15:09:08 2022] DELETE /identity/v3/projects/68969c3e05e9433dbac7ff643427c5f4 => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:09:08.835060 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb271989-627e-461b-ae04-1ac43abd4e1c admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:08.851406 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-bb271989-627e-461b-ae04-1ac43abd4e1c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:09:08.940205 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:09:09.026917 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 870/1721] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:09:08 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 359 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:09:09.039386 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8922a50e-a142-41a1-8ef6-051064c77dad None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:09.040310 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8922a50e-a142-41a1-8ef6-051064c77dad None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:09.040927 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8922a50e-a142-41a1-8ef6-051064c77dad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:09.133307 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb271989-627e-461b-ae04-1ac43abd4e1c admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:09.150201 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96879e2-03fe-4d31-87e3-dfb97af513ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:09.150489 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-8922a50e-a142-41a1-8ef6-051064c77dad None None] MFA Rules not processed for user `0156a2f6f3794f479e76b85bdbbe5c26`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:09:09.151115 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96879e2-03fe-4d31-87e3-dfb97af513ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:09.151403 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96879e2-03fe-4d31-87e3-dfb97af513ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:09.154876 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96879e2-03fe-4d31-87e3-dfb97af513ed admin admin] PATH_INFO: `/v3/users/fe5f60f71d5649bc8449be91da9127e9` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:09.155899 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-8922a50e-a142-41a1-8ef6-051064c77dad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:09:09.159619 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96879e2-03fe-4d31-87e3-dfb97af513ed admin admin] RBAC: Authorizing `identity:delete_user(user_id=fe5f60f71d5649bc8449be91da9127e9)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:09.167669 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96879e2-03fe-4d31-87e3-dfb97af513ed admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:09.431662 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-d96879e2-03fe-4d31-87e3-dfb97af513ed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fe5f60f71d5649bc8449be91da9127e9'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:09.447993 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-d96879e2-03fe-4d31-87e3-dfb97af513ed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fe5f60f71d5649bc8449be91da9127e9'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:09.449349 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-d96879e2-03fe-4d31-87e3-dfb97af513ed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fe5f60f71d5649bc8449be91da9127e9'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:09.481893 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-d96879e2-03fe-4d31-87e3-dfb97af513ed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fe5f60f71d5649bc8449be91da9127e9'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:09.503822 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-d96879e2-03fe-4d31-87e3-dfb97af513ed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fe5f60f71d5649bc8449be91da9127e9'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:09.507576 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 871/1722] 10.222.0.75 () {58 vars in 1015 bytes} [Thu May 26 15:09:09 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 471 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:09:09.509706 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8922a50e-a142-41a1-8ef6-051064c77dad None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:09.526160 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 852/1723] 10.222.0.46 () {58 vars in 1223 bytes} [Thu May 26 15:09:08 2022] DELETE /identity/v3/users/fe5f60f71d5649bc8449be91da9127e9 => generated 0 bytes in 690 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:09:09.526160 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f79ba3-8b2f-455c-bf3f-d5c84de90c6c None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:09.526160 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f79ba3-8b2f-455c-bf3f-d5c84de90c6c None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:09.526160 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f79ba3-8b2f-455c-bf3f-d5c84de90c6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:09.531476 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-8922a50e-a142-41a1-8ef6-051064c77dad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:09:09.562883 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:09:09.581791 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-f5f79ba3-8b2f-455c-bf3f-d5c84de90c6c None None] MFA Rules not processed for user `2ce5b5d5a12743bc80a4b191c24ff52f`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:09:09.584116 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-f5f79ba3-8b2f-455c-bf3f-d5c84de90c6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:09:09.808231 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 853/1724] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:09:09 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:09:09.811669 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f79ba3-8b2f-455c-bf3f-d5c84de90c6c None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:09.820404 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-f5f79ba3-8b2f-455c-bf3f-d5c84de90c6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:09:09.952259 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:09:09.981595 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8922a50e-a142-41a1-8ef6-051064c77dad None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:09.990709 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f79ba3-8b2f-455c-bf3f-d5c84de90c6c None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:09.995995 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfc6237-032b-487b-bd66-5d92cb8cd3e0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0156a2f6f3794f479e76b85bdbbe5c26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:09.996916 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfc6237-032b-487b-bd66-5d92cb8cd3e0 service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:09.997525 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfc6237-032b-487b-bd66-5d92cb8cd3e0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:09.998614 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfc6237-032b-487b-bd66-5d92cb8cd3e0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:10.019762 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfc6237-032b-487b-bd66-5d92cb8cd3e0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:10.023214 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfc6237-032b-487b-bd66-5d92cb8cd3e0 service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:10.043953 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 854/1725] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:09:09 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 15:09:10.159374 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfc6237-032b-487b-bd66-5d92cb8cd3e0 service neutron] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:10.164909 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-7dfc6237-032b-487b-bd66-5d92cb8cd3e0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:09:10.226801 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c418a1-d38f-412f-b884-3dbe6d0ee76e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6639353c70449d1b3d10cd6487c3b00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:10.227492 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c418a1-d38f-412f-b884-3dbe6d0ee76e service nova] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:10.227840 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c418a1-d38f-412f-b884-3dbe6d0ee76e service nova] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:10.231253 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c418a1-d38f-412f-b884-3dbe6d0ee76e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:10.242467 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-83c418a1-d38f-412f-b884-3dbe6d0ee76e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:09:10.392315 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfc6237-032b-487b-bd66-5d92cb8cd3e0 service neutron] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:10.412441 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08456c49-6164-4a64-ba44-cb645daeceb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:10.413961 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08456c49-6164-4a64-ba44-cb645daeceb5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:10.417183 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08456c49-6164-4a64-ba44-cb645daeceb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:10.418162 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08456c49-6164-4a64-ba44-cb645daeceb5 admin admin] PATH_INFO: `/v3/projects/e4e82e4c96364f3e85f5c8673fdf5d91` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:10.424423 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08456c49-6164-4a64-ba44-cb645daeceb5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e4e82e4c96364f3e85f5c8673fdf5d91)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:10.435887 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08456c49-6164-4a64-ba44-cb645daeceb5 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:10.503558 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c418a1-d38f-412f-b884-3dbe6d0ee76e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:10.505257 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c418a1-d38f-412f-b884-3dbe6d0ee76e service nova] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:10.550868 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 872/1726] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:09:09 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 1042 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 15:09:10.589771 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-08456c49-6164-4a64-ba44-cb645daeceb5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e4e82e4c96364f3e85f5c8673fdf5d91', 'request_id': 'req-08456c49-6164-4a64-ba44-cb645daeceb5'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:10.618995 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-08456c49-6164-4a64-ba44-cb645daeceb5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e4e82e4c96364f3e85f5c8673fdf5d91', 'request_id': 'req-08456c49-6164-4a64-ba44-cb645daeceb5'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:10.645721 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-08456c49-6164-4a64-ba44-cb645daeceb5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e4e82e4c96364f3e85f5c8673fdf5d91', 'request_id': 'req-08456c49-6164-4a64-ba44-cb645daeceb5'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:10.696905 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-08456c49-6164-4a64-ba44-cb645daeceb5 admin admin] The token cache is being invalidate because project e4e82e4c96364f3e85f5c8673fdf5d91 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 15:09:10.699726 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-08456c49-6164-4a64-ba44-cb645daeceb5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:10.708128 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-08456c49-6164-4a64-ba44-cb645daeceb5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:10.711137 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 855/1727] 10.222.0.46 () {58 vars in 1232 bytes} [Thu May 26 15:09:10 2022] DELETE /identity/v3/projects/e4e82e4c96364f3e85f5c8673fdf5d91 => generated 0 bytes in 554 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 15:09:10.717837 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2124a00e-9489-4905-9802-37cd6e94b1b7 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:10.718153 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2124a00e-9489-4905-9802-37cd6e94b1b7 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:10.718429 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2124a00e-9489-4905-9802-37cd6e94b1b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:10.788947 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-2124a00e-9489-4905-9802-37cd6e94b1b7 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:09:10.790683 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-2124a00e-9489-4905-9802-37cd6e94b1b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:09:10.977474 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08456c49-6164-4a64-ba44-cb645daeceb5 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:10.979493 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-08456c49-6164-4a64-ba44-cb645daeceb5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:09:11.093258 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:09:11.146659 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 873/1728] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:09:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 429 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:09:11.156074 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61f73e7-f6c2-45c9-8a9a-a12a11403aa8 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:11.156390 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61f73e7-f6c2-45c9-8a9a-a12a11403aa8 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:11.158893 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61f73e7-f6c2-45c9-8a9a-a12a11403aa8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:11.291831 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-b61f73e7-f6c2-45c9-8a9a-a12a11403aa8 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:09:11.303392 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-b61f73e7-f6c2-45c9-8a9a-a12a11403aa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:09:11.337791 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:09:11.352313 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 874/1729] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:09:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:09:11.356245 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61f73e7-f6c2-45c9-8a9a-a12a11403aa8 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:11.358970 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08456c49-6164-4a64-ba44-cb645daeceb5 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:11.383681 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07abc25-d499-40b8-bf39-f6e1341c5971 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6639353c70449d1b3d10cd6487c3b00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:11.384502 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07abc25-d499-40b8-bf39-f6e1341c5971 service nova] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:11.384860 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07abc25-d499-40b8-bf39-f6e1341c5971 service nova] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:11.385285 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c07abc25-d499-40b8-bf39-f6e1341c5971 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:11.389317 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-c07abc25-d499-40b8-bf39-f6e1341c5971 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:09:11.398024 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61f73e7-f6c2-45c9-8a9a-a12a11403aa8 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:11.434602 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bc1e18-e52c-445e-afaf-ae9fcebccdf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:11.438975 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bc1e18-e52c-445e-afaf-ae9fcebccdf8 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:11.439422 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bc1e18-e52c-445e-afaf-ae9fcebccdf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:11.439800 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93bc1e18-e52c-445e-afaf-ae9fcebccdf8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:11.445650 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93bc1e18-e52c-445e-afaf-ae9fcebccdf8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 15:09:11.446193 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93bc1e18-e52c-445e-afaf-ae9fcebccdf8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:11.461381 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93bc1e18-e52c-445e-afaf-ae9fcebccdf8 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:11.545913 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 875/1730] 10.222.0.46 () {58 vars in 1152 bytes} [Thu May 26 15:09:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 15:09:11.547572 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bc1e18-e52c-445e-afaf-ae9fcebccdf8 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:11.592442 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93bc1e18-e52c-445e-afaf-ae9fcebccdf8 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:11.625950 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69385a16-96b9-4cee-badd-37cb32dbf65d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:11.626881 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69385a16-96b9-4cee-badd-37cb32dbf65d admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:11.627279 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69385a16-96b9-4cee-badd-37cb32dbf65d admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:11.627626 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69385a16-96b9-4cee-badd-37cb32dbf65d admin admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:11.628616 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69385a16-96b9-4cee-badd-37cb32dbf65d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:11.635568 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69385a16-96b9-4cee-badd-37cb32dbf65d admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:11.645003 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07abc25-d499-40b8-bf39-f6e1341c5971 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:11.651929 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c07abc25-d499-40b8-bf39-f6e1341c5971 service nova] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:11.724446 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 856/1731] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:09:10 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 748 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 15:09:11.771909 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 876/1732] 10.222.0.46 () {60 vars in 1152 bytes} [Thu May 26 15:09:11 2022] POST /identity/v3/projects => generated 383 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 15:09:11.772558 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07abc25-d499-40b8-bf39-f6e1341c5971 service nova] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:11.848934 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c07abc25-d499-40b8-bf39-f6e1341c5971 service nova] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:12.080254 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dee2474-58a1-43f0-825d-2e80c798f92c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:12.081046 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dee2474-58a1-43f0-825d-2e80c798f92c admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:12.081422 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dee2474-58a1-43f0-825d-2e80c798f92c admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:12.081784 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dee2474-58a1-43f0-825d-2e80c798f92c admin admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:12.088332 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dee2474-58a1-43f0-825d-2e80c798f92c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:12.090190 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dee2474-58a1-43f0-825d-2e80c798f92c admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:12.158004 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 857/1733] 10.222.0.46 () {60 vars in 1143 bytes} [Thu May 26 15:09:11 2022] POST /identity/v3/users => generated 292 bytes in 384 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 15:09:12.161062 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69385a16-96b9-4cee-badd-37cb32dbf65d admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:12.184288 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69385a16-96b9-4cee-badd-37cb32dbf65d admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:12.191835 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772f6d66-6f66-4fff-9f86-21239568f59d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:12.193089 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772f6d66-6f66-4fff-9f86-21239568f59d admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:12.193778 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772f6d66-6f66-4fff-9f86-21239568f59d admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:12.194242 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772f6d66-6f66-4fff-9f86-21239568f59d admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:12.196509 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772f6d66-6f66-4fff-9f86-21239568f59d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:12.201349 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772f6d66-6f66-4fff-9f86-21239568f59d admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:12.231704 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 877/1734] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 15:09:12 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 15:09:12.232787 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dee2474-58a1-43f0-825d-2e80c798f92c admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:12.253103 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dee2474-58a1-43f0-825d-2e80c798f92c admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:12.282510 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0495569-2f5b-40cc-bdcf-54011c8bef61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:12.283400 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0495569-2f5b-40cc-bdcf-54011c8bef61 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:12.283984 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0495569-2f5b-40cc-bdcf-54011c8bef61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:12.284307 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0495569-2f5b-40cc-bdcf-54011c8bef61 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:12.285072 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0495569-2f5b-40cc-bdcf-54011c8bef61 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:12.286823 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0495569-2f5b-40cc-bdcf-54011c8bef61 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:12.309414 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 858/1735] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 15:09:12 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 15:09:12.316457 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772f6d66-6f66-4fff-9f86-21239568f59d admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:12.381022 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772f6d66-6f66-4fff-9f86-21239568f59d admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:12.390930 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7cefe7-8d7a-4b8f-b526-10fabc4e8f96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:12.393120 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7cefe7-8d7a-4b8f-b526-10fabc4e8f96 admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:12.393631 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7cefe7-8d7a-4b8f-b526-10fabc4e8f96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:12.394157 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7cefe7-8d7a-4b8f-b526-10fabc4e8f96 admin admin] PATH_INFO: `/v3/projects/2376a926511f4759bd0c9513f7edf91e/users/f3ebb826ca324603a2912f78c5d27b38/roles/d4088d1845a24fbd9bd8975ce0b1d3d0` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:12.472482 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7cefe7-8d7a-4b8f-b526-10fabc4e8f96 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2376a926511f4759bd0c9513f7edf91e, user_id=f3ebb826ca324603a2912f78c5d27b38, role_id=d4088d1845a24fbd9bd8975ce0b1d3d0)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:12.491620 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7cefe7-8d7a-4b8f-b526-10fabc4e8f96 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:12.562130 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 878/1736] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 15:09:12 2022] PUT /identity/v3/projects/2376a926511f4759bd0c9513f7edf91e/users/f3ebb826ca324603a2912f78c5d27b38/roles/d4088d1845a24fbd9bd8975ce0b1d3d0 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:09:12.574306 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434b0868-4165-48ea-9bf7-1d181e710509 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:12.574306 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434b0868-4165-48ea-9bf7-1d181e710509 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:12.574857 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434b0868-4165-48ea-9bf7-1d181e710509 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:12.710940 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-434b0868-4165-48ea-9bf7-1d181e710509 None None] MFA Rules not processed for user `f3ebb826ca324603a2912f78c5d27b38`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:09:12.720910 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-434b0868-4165-48ea-9bf7-1d181e710509 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:09:12.967298 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 859/1737] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:09:12 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 395 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:09:12.972107 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7cefe7-8d7a-4b8f-b526-10fabc4e8f96 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:13.197737 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7cefe7-8d7a-4b8f-b526-10fabc4e8f96 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:13.205553 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37396244-d2da-47ef-8037-2473a1c4b21d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:13.206480 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37396244-d2da-47ef-8037-2473a1c4b21d admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:13.206926 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37396244-d2da-47ef-8037-2473a1c4b21d admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:13.207366 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37396244-d2da-47ef-8037-2473a1c4b21d admin admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:13.208237 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37396244-d2da-47ef-8037-2473a1c4b21d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:13.214349 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37396244-d2da-47ef-8037-2473a1c4b21d admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:13.234180 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c84c8f5-7f0a-4333-a087-6ab839ae7466 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:13.234844 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c84c8f5-7f0a-4333-a087-6ab839ae7466 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:13.235346 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c84c8f5-7f0a-4333-a087-6ab839ae7466 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:13.308618 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:09:13.312695 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 879/1738] 10.222.0.46 () {60 vars in 1152 bytes} [Thu May 26 15:09:12 2022] POST /identity/v3/projects => generated 387 bytes in 341 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 15:09:13.317420 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37396244-d2da-47ef-8037-2473a1c4b21d admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:13.350749 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-7c84c8f5-7f0a-4333-a087-6ab839ae7466 None None] MFA Rules not processed for user `e95b6425286f4235b9d1802acddac4b8`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:09:13.352925 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-7c84c8f5-7f0a-4333-a087-6ab839ae7466 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:09:13.477075 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:09:13.502888 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 860/1739] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:09:13 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 270 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 15:09:13.504605 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c84c8f5-7f0a-4333-a087-6ab839ae7466 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:13.505498 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37396244-d2da-47ef-8037-2473a1c4b21d admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:13.531358 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf163a9-2860-4bb2-a56b-a50f70664d2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:13.531883 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf163a9-2860-4bb2-a56b-a50f70664d2f admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:13.532260 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf163a9-2860-4bb2-a56b-a50f70664d2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:13.532589 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf163a9-2860-4bb2-a56b-a50f70664d2f admin admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:13.533487 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf163a9-2860-4bb2-a56b-a50f70664d2f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:13.535819 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf163a9-2860-4bb2-a56b-a50f70664d2f admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:13.598097 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 880/1740] 10.222.0.46 () {60 vars in 1143 bytes} [Thu May 26 15:09:13 2022] POST /identity/v3/users => generated 300 bytes in 281 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 15:09:13.617273 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf163a9-2860-4bb2-a56b-a50f70664d2f admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:13.676283 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf163a9-2860-4bb2-a56b-a50f70664d2f admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:13.698277 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3284ee-6a44-4f6f-a466-8d02c1963755 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:13.699571 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3284ee-6a44-4f6f-a466-8d02c1963755 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:13.699571 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3284ee-6a44-4f6f-a466-8d02c1963755 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:13.699571 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e3284ee-6a44-4f6f-a466-8d02c1963755 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:13.701722 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3284ee-6a44-4f6f-a466-8d02c1963755 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:13.713804 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e3284ee-6a44-4f6f-a466-8d02c1963755 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:13.737366 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 881/1741] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 15:09:13 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 15:09:13.743512 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3284ee-6a44-4f6f-a466-8d02c1963755 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:13.779409 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c84c8f5-7f0a-4333-a087-6ab839ae7466 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:13.784791 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e3284ee-6a44-4f6f-a466-8d02c1963755 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:13.807106 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8020217f-7748-4d28-adcb-4eb2191d966d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6639353c70449d1b3d10cd6487c3b00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:13.807956 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16a5731-4638-4b5a-bbdb-e58e9507ec9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:13.808924 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16a5731-4638-4b5a-bbdb-e58e9507ec9f admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:13.809214 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16a5731-4638-4b5a-bbdb-e58e9507ec9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:13.809559 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b16a5731-4638-4b5a-bbdb-e58e9507ec9f admin admin] PATH_INFO: `/v3/projects/3a3c1e84727d40e28db1d10900ca0f40/users/1e31b26f2351467bbecbb4afabcaefe6/roles/a2e551773cbd4df084020676865b8222` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:13.811579 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8020217f-7748-4d28-adcb-4eb2191d966d service nova] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:13.812099 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8020217f-7748-4d28-adcb-4eb2191d966d service nova] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:13.813964 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8020217f-7748-4d28-adcb-4eb2191d966d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:13.841053 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8020217f-7748-4d28-adcb-4eb2191d966d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:13.845893 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8020217f-7748-4d28-adcb-4eb2191d966d service nova] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:13.859408 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16a5731-4638-4b5a-bbdb-e58e9507ec9f admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a3c1e84727d40e28db1d10900ca0f40, user_id=1e31b26f2351467bbecbb4afabcaefe6, role_id=a2e551773cbd4df084020676865b8222)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:13.862263 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b16a5731-4638-4b5a-bbdb-e58e9507ec9f admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:13.871788 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 861/1742] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:09:13 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 15:09:13.943056 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 882/1743] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 15:09:13 2022] PUT /identity/v3/projects/3a3c1e84727d40e28db1d10900ca0f40/users/1e31b26f2351467bbecbb4afabcaefe6/roles/a2e551773cbd4df084020676865b8222 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:09:13.944295 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8020217f-7748-4d28-adcb-4eb2191d966d service nova] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:13.967988 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8020217f-7748-4d28-adcb-4eb2191d966d service nova] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:14.174104 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8f8d4d-ee25-41f2-a143-1694ef8a6d33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:14.174814 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8f8d4d-ee25-41f2-a143-1694ef8a6d33 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:14.175086 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8f8d4d-ee25-41f2-a143-1694ef8a6d33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:14.175362 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8f8d4d-ee25-41f2-a143-1694ef8a6d33 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:14.176114 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8f8d4d-ee25-41f2-a143-1694ef8a6d33 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:14.177822 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8f8d4d-ee25-41f2-a143-1694ef8a6d33 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:14.221216 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 862/1744] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 15:09:13 2022] GET /identity/v3/roles => generated 1420 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 15:09:14.225220 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16a5731-4638-4b5a-bbdb-e58e9507ec9f admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:14.252300 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b16a5731-4638-4b5a-bbdb-e58e9507ec9f admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:14.273063 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0b4b18-ee55-4862-9767-80f21d7a1d3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:14.275385 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0b4b18-ee55-4862-9767-80f21d7a1d3d admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:14.276857 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0b4b18-ee55-4862-9767-80f21d7a1d3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:14.277295 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0b4b18-ee55-4862-9767-80f21d7a1d3d admin admin] PATH_INFO: `/v3/projects/3a3c1e84727d40e28db1d10900ca0f40/users/1e31b26f2351467bbecbb4afabcaefe6/roles/a2e551773cbd4df084020676865b8222` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:14.286227 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0b4b18-ee55-4862-9767-80f21d7a1d3d admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a3c1e84727d40e28db1d10900ca0f40, user_id=1e31b26f2351467bbecbb4afabcaefe6, role_id=a2e551773cbd4df084020676865b8222)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:14.289310 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0b4b18-ee55-4862-9767-80f21d7a1d3d admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:14.324396 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 883/1745] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 15:09:14 2022] PUT /identity/v3/projects/3a3c1e84727d40e28db1d10900ca0f40/users/1e31b26f2351467bbecbb4afabcaefe6/roles/a2e551773cbd4df084020676865b8222 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 15:09:14.341125 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431884fe-bbcd-49f2-aa87-725318403501 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:14.342373 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431884fe-bbcd-49f2-aa87-725318403501 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:14.342868 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431884fe-bbcd-49f2-aa87-725318403501 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:14.419773 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-431884fe-bbcd-49f2-aa87-725318403501 None None] MFA Rules not processed for user `1e31b26f2351467bbecbb4afabcaefe6`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:09:14.421596 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-431884fe-bbcd-49f2-aa87-725318403501 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:09:14.443479 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:09:14.635796 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 863/1746] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:09:14 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 296 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:09:14.661981 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0b4b18-ee55-4862-9767-80f21d7a1d3d admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:14.667983 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-eb0b4b18-ee55-4862-9767-80f21d7a1d3d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:09:14.930633 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0b4b18-ee55-4862-9767-80f21d7a1d3d admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:14.954701 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35182811-6523-4373-8b60-99607fdf2e32 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5b5b35614a547d783e6e77ddb8c1c03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:14.955135 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35182811-6523-4373-8b60-99607fdf2e32 service cinder] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:14.955415 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35182811-6523-4373-8b60-99607fdf2e32 service cinder] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:14.955884 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35182811-6523-4373-8b60-99607fdf2e32 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:14.999792 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35182811-6523-4373-8b60-99607fdf2e32 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:15.001385 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35182811-6523-4373-8b60-99607fdf2e32 service cinder] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:15.034189 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 884/1747] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:09:14 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 15:09:16.776828 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba4e1d8-b9f1-4f12-aceb-36efc2b603fa None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:16.777315 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba4e1d8-b9f1-4f12-aceb-36efc2b603fa None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:16.777388 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba4e1d8-b9f1-4f12-aceb-36efc2b603fa None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:16.779454 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 864/1748] 10.222.0.46 () {54 vars in 921 bytes} [Thu May 26 15:09:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 15:09:16.818324 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b29540c-6321-4357-b280-7a0bab6b5342 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:16.819432 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b29540c-6321-4357-b280-7a0bab6b5342 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:16.819913 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b29540c-6321-4357-b280-7a0bab6b5342 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:16.894743 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-9b29540c-6321-4357-b280-7a0bab6b5342 None None] MFA Rules not processed for user `d6639353c70449d1b3d10cd6487c3b00`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:09:16.897084 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-9b29540c-6321-4357-b280-7a0bab6b5342 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:09:17.311233 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 885/1749] 10.222.0.46 () {58 vars in 1018 bytes} [Thu May 26 15:09:16 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 494 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:09:17.720154 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba4e1d8-b9f1-4f12-aceb-36efc2b603fa None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:17.722159 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-aba4e1d8-b9f1-4f12-aceb-36efc2b603fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:09:17.803265 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba4e1d8-b9f1-4f12-aceb-36efc2b603fa None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:17.830393 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7037872a-b4fb-41fe-b09b-ddbb31325088 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6639353c70449d1b3d10cd6487c3b00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:17.832218 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7037872a-b4fb-41fe-b09b-ddbb31325088 service nova] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:17.834449 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7037872a-b4fb-41fe-b09b-ddbb31325088 service nova] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:17.835836 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7037872a-b4fb-41fe-b09b-ddbb31325088 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:17.877763 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7037872a-b4fb-41fe-b09b-ddbb31325088 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:17.879761 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7037872a-b4fb-41fe-b09b-ddbb31325088 service nova] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:17.932073 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 865/1750] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:09:17 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 15:09:18.028506 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b29540c-6321-4357-b280-7a0bab6b5342 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:18.198808 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:09:18.333794 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b29540c-6321-4357-b280-7a0bab6b5342 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:18.360660 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08aeb550-f35f-4fd1-aed4-c43d83b8250c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:18.362088 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08aeb550-f35f-4fd1-aed4-c43d83b8250c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:18.363443 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08aeb550-f35f-4fd1-aed4-c43d83b8250c admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:18.364539 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08aeb550-f35f-4fd1-aed4-c43d83b8250c admin admin] PATH_INFO: `/v3/users/f3ebb826ca324603a2912f78c5d27b38` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:18.368359 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08aeb550-f35f-4fd1-aed4-c43d83b8250c admin admin] RBAC: Authorizing `identity:delete_user(user_id=f3ebb826ca324603a2912f78c5d27b38)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:18.377016 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08aeb550-f35f-4fd1-aed4-c43d83b8250c admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:18.580207 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-08aeb550-f35f-4fd1-aed4-c43d83b8250c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f3ebb826ca324603a2912f78c5d27b38'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:18.602763 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-08aeb550-f35f-4fd1-aed4-c43d83b8250c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f3ebb826ca324603a2912f78c5d27b38'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:18.614901 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-08aeb550-f35f-4fd1-aed4-c43d83b8250c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f3ebb826ca324603a2912f78c5d27b38'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:18.688544 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-08aeb550-f35f-4fd1-aed4-c43d83b8250c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f3ebb826ca324603a2912f78c5d27b38'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:18.748401 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-08aeb550-f35f-4fd1-aed4-c43d83b8250c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f3ebb826ca324603a2912f78c5d27b38'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:18.868530 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 886/1751] 10.222.0.46 () {58 vars in 1223 bytes} [Thu May 26 15:09:18 2022] DELETE /identity/v3/users/f3ebb826ca324603a2912f78c5d27b38 => generated 0 bytes in 840 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:09:18.876616 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babe7d50-90b8-484a-ab1c-b2c5c2360024 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:18.877752 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babe7d50-90b8-484a-ab1c-b2c5c2360024 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:18.879654 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babe7d50-90b8-484a-ab1c-b2c5c2360024 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:18.975919 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-babe7d50-90b8-484a-ab1c-b2c5c2360024 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:09:18.977228 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08aeb550-f35f-4fd1-aed4-c43d83b8250c admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:18.977653 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-babe7d50-90b8-484a-ab1c-b2c5c2360024 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:09:18.979200 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-08aeb550-f35f-4fd1-aed4-c43d83b8250c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:09:19.230101 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 866/1752] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:09:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 355 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:09:19.301329 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08aeb550-f35f-4fd1-aed4-c43d83b8250c admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:19.323427 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba610c55-d950-42a1-a8bf-e1f019571b7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:19.337984 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba610c55-d950-42a1-a8bf-e1f019571b7c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:19.338559 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba610c55-d950-42a1-a8bf-e1f019571b7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:19.339001 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba610c55-d950-42a1-a8bf-e1f019571b7c admin admin] PATH_INFO: `/v3/users/5bf28ae4216140e9a2bd5fc2d90ab48b` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:19.340714 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba610c55-d950-42a1-a8bf-e1f019571b7c admin admin] RBAC: Authorizing `identity:delete_user(user_id=5bf28ae4216140e9a2bd5fc2d90ab48b)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:19.344309 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba610c55-d950-42a1-a8bf-e1f019571b7c admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:19.361494 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babe7d50-90b8-484a-ab1c-b2c5c2360024 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:19.363168 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-babe7d50-90b8-484a-ab1c-b2c5c2360024 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:09:19.546011 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-ba610c55-d950-42a1-a8bf-e1f019571b7c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5bf28ae4216140e9a2bd5fc2d90ab48b'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:19.561281 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-ba610c55-d950-42a1-a8bf-e1f019571b7c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5bf28ae4216140e9a2bd5fc2d90ab48b'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:19.563335 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-ba610c55-d950-42a1-a8bf-e1f019571b7c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5bf28ae4216140e9a2bd5fc2d90ab48b'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:19.565193 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=80,keystone:DELETE=18,keystone:INSERT=2 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:09:19.633565 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-ba610c55-d950-42a1-a8bf-e1f019571b7c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5bf28ae4216140e9a2bd5fc2d90ab48b'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:19.650096 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-ba610c55-d950-42a1-a8bf-e1f019571b7c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5bf28ae4216140e9a2bd5fc2d90ab48b'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:19.659092 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babe7d50-90b8-484a-ab1c-b2c5c2360024 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:19.670742 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 887/1753] 10.222.0.46 () {58 vars in 1223 bytes} [Thu May 26 15:09:18 2022] DELETE /identity/v3/users/5bf28ae4216140e9a2bd5fc2d90ab48b => generated 0 bytes in 691 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:09:19.672800 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a582f101-2e27-4370-9547-2a2b37a0fe25 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0156a2f6f3794f479e76b85bdbbe5c26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:19.673274 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1778b5b0-bd62-4cf2-a15f-e385cde5a2dc None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:19.673714 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1778b5b0-bd62-4cf2-a15f-e385cde5a2dc None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:19.673957 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1778b5b0-bd62-4cf2-a15f-e385cde5a2dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:19.674891 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a582f101-2e27-4370-9547-2a2b37a0fe25 service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:19.676884 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a582f101-2e27-4370-9547-2a2b37a0fe25 service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:19.677373 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a582f101-2e27-4370-9547-2a2b37a0fe25 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:19.690425 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-a582f101-2e27-4370-9547-2a2b37a0fe25 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:09:19.722219 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-1778b5b0-bd62-4cf2-a15f-e385cde5a2dc None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:09:19.724169 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-1778b5b0-bd62-4cf2-a15f-e385cde5a2dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:09:19.735683 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a582f101-2e27-4370-9547-2a2b37a0fe25 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:19.737346 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a582f101-2e27-4370-9547-2a2b37a0fe25 service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:19.797422 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 867/1754] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:09:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 436 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 15:09:20.007482 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 888/1755] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:09:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 336 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:09:20.075153 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a582f101-2e27-4370-9547-2a2b37a0fe25 service neutron] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:20.078742 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-a582f101-2e27-4370-9547-2a2b37a0fe25 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:09:20.104502 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a582f101-2e27-4370-9547-2a2b37a0fe25 service neutron] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:20.321384 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:09:20.410361 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4ca6fa-884e-49a0-a80d-ddf1fde1e209 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0156a2f6f3794f479e76b85bdbbe5c26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:20.411116 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4ca6fa-884e-49a0-a80d-ddf1fde1e209 service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:20.411408 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4ca6fa-884e-49a0-a80d-ddf1fde1e209 service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:20.411687 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4ca6fa-884e-49a0-a80d-ddf1fde1e209 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:20.452027 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4ca6fa-884e-49a0-a80d-ddf1fde1e209 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:20.466172 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4ca6fa-884e-49a0-a80d-ddf1fde1e209 service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:20.468857 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dbd999-2961-4892-89cb-e3a9e564db7b None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:20.469582 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dbd999-2961-4892-89cb-e3a9e564db7b None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:20.470212 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dbd999-2961-4892-89cb-e3a9e564db7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:20.494391 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 868/1756] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:09:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 15:09:20.531094 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-d4dbd999-2961-4892-89cb-e3a9e564db7b None None] MFA Rules not processed for user `64aeada36ebe460e9969f1f048651e6d`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:09:20.537107 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-d4dbd999-2961-4892-89cb-e3a9e564db7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:09:20.571482 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4ca6fa-884e-49a0-a80d-ddf1fde1e209 service neutron] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:20.573187 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-7e4ca6fa-884e-49a0-a80d-ddf1fde1e209 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:09:20.601754 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4ca6fa-884e-49a0-a80d-ddf1fde1e209 service neutron] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:20.628791 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0526b72e-d528-4941-ab9c-9bdf4632fff1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:20.630018 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0526b72e-d528-4941-ab9c-9bdf4632fff1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:20.630464 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0526b72e-d528-4941-ab9c-9bdf4632fff1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:20.630895 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0526b72e-d528-4941-ab9c-9bdf4632fff1 admin admin] PATH_INFO: `/v3/users/4214e30903b543ada0a7edad0cf1e8ef` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:20.632534 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0526b72e-d528-4941-ab9c-9bdf4632fff1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4214e30903b543ada0a7edad0cf1e8ef)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:20.635096 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0526b72e-d528-4941-ab9c-9bdf4632fff1 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:20.768307 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 889/1757] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:09:20 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 301 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:09:20.771259 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-0526b72e-d528-4941-ab9c-9bdf4632fff1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4214e30903b543ada0a7edad0cf1e8ef'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:20.789258 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-0526b72e-d528-4941-ab9c-9bdf4632fff1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4214e30903b543ada0a7edad0cf1e8ef'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:20.790250 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-0526b72e-d528-4941-ab9c-9bdf4632fff1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4214e30903b543ada0a7edad0cf1e8ef'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:20.814488 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-0526b72e-d528-4941-ab9c-9bdf4632fff1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4214e30903b543ada0a7edad0cf1e8ef'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:20.845679 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-0526b72e-d528-4941-ab9c-9bdf4632fff1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4214e30903b543ada0a7edad0cf1e8ef'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:20.863314 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 869/1758] 10.222.0.46 () {58 vars in 1223 bytes} [Thu May 26 15:09:20 2022] DELETE /identity/v3/users/4214e30903b543ada0a7edad0cf1e8ef => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:09:20.867139 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9afe73-4708-436f-9732-039863a8f77b None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:20.867600 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9afe73-4708-436f-9732-039863a8f77b None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:20.868022 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9afe73-4708-436f-9732-039863a8f77b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:20.902072 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-7b9afe73-4708-436f-9732-039863a8f77b None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:09:20.903487 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-7b9afe73-4708-436f-9732-039863a8f77b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:09:21.056231 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 890/1759] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:09:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 15:09:21.088545 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0526b72e-d528-4941-ab9c-9bdf4632fff1 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:21.090151 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-0526b72e-d528-4941-ab9c-9bdf4632fff1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:09:21.283685 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0526b72e-d528-4941-ab9c-9bdf4632fff1 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:21.292133 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1820ecbc-2c0b-4680-987b-a3a3eff71b86 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0156a2f6f3794f479e76b85bdbbe5c26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:21.293519 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1820ecbc-2c0b-4680-987b-a3a3eff71b86 service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:21.294193 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1820ecbc-2c0b-4680-987b-a3a3eff71b86 service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:21.294665 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1820ecbc-2c0b-4680-987b-a3a3eff71b86 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:21.313924 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1820ecbc-2c0b-4680-987b-a3a3eff71b86 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:21.317348 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1820ecbc-2c0b-4680-987b-a3a3eff71b86 service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:21.350352 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 870/1760] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:09:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 262 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 15:09:21.776248 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9afe73-4708-436f-9732-039863a8f77b None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:21.778945 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-7b9afe73-4708-436f-9732-039863a8f77b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:09:21.803000 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9afe73-4708-436f-9732-039863a8f77b None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:21.823677 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8984f3-b43c-45fa-ad66-59d425b448e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:21.825352 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8984f3-b43c-45fa-ad66-59d425b448e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:21.825352 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8984f3-b43c-45fa-ad66-59d425b448e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:21.825352 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8984f3-b43c-45fa-ad66-59d425b448e7 admin admin] PATH_INFO: `/v3/projects/2376a926511f4759bd0c9513f7edf91e` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:21.827798 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8984f3-b43c-45fa-ad66-59d425b448e7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2376a926511f4759bd0c9513f7edf91e)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:21.831126 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8984f3-b43c-45fa-ad66-59d425b448e7 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:21.888081 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:09:21.971674 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3bf439-d263-4019-8e36-2b85ade008ef None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:21.972801 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3bf439-d263-4019-8e36-2b85ade008ef None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:21.975926 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3bf439-d263-4019-8e36-2b85ade008ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:22.029737 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-8d3bf439-d263-4019-8e36-2b85ade008ef None None] MFA Rules not processed for user `e95b6425286f4235b9d1802acddac4b8`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:09:22.029989 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-8e8984f3-b43c-45fa-ad66-59d425b448e7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2376a926511f4759bd0c9513f7edf91e', 'request_id': 'req-8e8984f3-b43c-45fa-ad66-59d425b448e7'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:22.031368 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-8d3bf439-d263-4019-8e36-2b85ade008ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:09:22.057584 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-8e8984f3-b43c-45fa-ad66-59d425b448e7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2376a926511f4759bd0c9513f7edf91e', 'request_id': 'req-8e8984f3-b43c-45fa-ad66-59d425b448e7'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:22.075643 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-8e8984f3-b43c-45fa-ad66-59d425b448e7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2376a926511f4759bd0c9513f7edf91e', 'request_id': 'req-8e8984f3-b43c-45fa-ad66-59d425b448e7'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:22.104825 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-8e8984f3-b43c-45fa-ad66-59d425b448e7 admin admin] The token cache is being invalidate because project 2376a926511f4759bd0c9513f7edf91e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 15:09:22.105409 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-8e8984f3-b43c-45fa-ad66-59d425b448e7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:22.107986 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-8e8984f3-b43c-45fa-ad66-59d425b448e7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:22.114370 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 891/1761] 10.222.0.46 () {58 vars in 1232 bytes} [Thu May 26 15:09:21 2022] DELETE /identity/v3/projects/2376a926511f4759bd0c9513f7edf91e => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:09:22.116919 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8984f3-b43c-45fa-ad66-59d425b448e7 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:22.118615 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-8e8984f3-b43c-45fa-ad66-59d425b448e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:09:22.148327 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:09:22.173741 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 871/1762] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:09:21 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:09:22.245736 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3bf439-d263-4019-8e36-2b85ade008ef None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:22.248470 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-8d3bf439-d263-4019-8e36-2b85ade008ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:09:22.276774 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8984f3-b43c-45fa-ad66-59d425b448e7 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:22.297143 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe213a4-1d09-4225-b051-962222731a06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:22.299240 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe213a4-1d09-4225-b051-962222731a06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:22.300111 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe213a4-1d09-4225-b051-962222731a06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:22.302298 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe213a4-1d09-4225-b051-962222731a06 admin admin] PATH_INFO: `/v3/users/1e31b26f2351467bbecbb4afabcaefe6` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:22.305211 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe213a4-1d09-4225-b051-962222731a06 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1e31b26f2351467bbecbb4afabcaefe6)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:22.311048 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe213a4-1d09-4225-b051-962222731a06 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:22.420727 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-cfe213a4-1d09-4225-b051-962222731a06 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1e31b26f2351467bbecbb4afabcaefe6'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:22.433782 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-cfe213a4-1d09-4225-b051-962222731a06 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1e31b26f2351467bbecbb4afabcaefe6'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:22.434423 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-cfe213a4-1d09-4225-b051-962222731a06 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1e31b26f2351467bbecbb4afabcaefe6'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:22.460012 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3bf439-d263-4019-8e36-2b85ade008ef None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:22.462700 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-cfe213a4-1d09-4225-b051-962222731a06 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1e31b26f2351467bbecbb4afabcaefe6'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:22.471912 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4e74b2-8125-42d1-b529-373d8aa7aa1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:22.474003 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4e74b2-8125-42d1-b529-373d8aa7aa1e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:22.474699 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4e74b2-8125-42d1-b529-373d8aa7aa1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:22.475142 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4e74b2-8125-42d1-b529-373d8aa7aa1e admin admin] PATH_INFO: `/v3/projects/5792798bb80e49e895262713d5368988` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:22.490135 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-cfe213a4-1d09-4225-b051-962222731a06 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1e31b26f2351467bbecbb4afabcaefe6'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:22.490503 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4e74b2-8125-42d1-b529-373d8aa7aa1e admin admin] RBAC: Authorizing `identity:delete_project(project_id=5792798bb80e49e895262713d5368988)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:22.490503 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4e74b2-8125-42d1-b529-373d8aa7aa1e admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:22.502994 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 892/1763] 10.222.0.46 () {58 vars in 1223 bytes} [Thu May 26 15:09:22 2022] DELETE /identity/v3/users/1e31b26f2351467bbecbb4afabcaefe6 => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:09:22.603608 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-6e4e74b2-8125-42d1-b529-373d8aa7aa1e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5792798bb80e49e895262713d5368988', 'request_id': 'req-6e4e74b2-8125-42d1-b529-373d8aa7aa1e'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:22.612297 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-6e4e74b2-8125-42d1-b529-373d8aa7aa1e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5792798bb80e49e895262713d5368988', 'request_id': 'req-6e4e74b2-8125-42d1-b529-373d8aa7aa1e'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:22.626133 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-6e4e74b2-8125-42d1-b529-373d8aa7aa1e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5792798bb80e49e895262713d5368988', 'request_id': 'req-6e4e74b2-8125-42d1-b529-373d8aa7aa1e'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:22.663818 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-6e4e74b2-8125-42d1-b529-373d8aa7aa1e admin admin] The token cache is being invalidate because project 5792798bb80e49e895262713d5368988 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 15:09:22.664336 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-6e4e74b2-8125-42d1-b529-373d8aa7aa1e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:22.665396 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-6e4e74b2-8125-42d1-b529-373d8aa7aa1e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:22.674825 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe213a4-1d09-4225-b051-962222731a06 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:22.677002 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 872/1764] 10.222.0.46 () {58 vars in 1232 bytes} [Thu May 26 15:09:22 2022] DELETE /identity/v3/projects/5792798bb80e49e895262713d5368988 => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:09:22.677392 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-cfe213a4-1d09-4225-b051-962222731a06 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:09:22.720451 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:09:22.806085 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe213a4-1d09-4225-b051-962222731a06 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:22.954990 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4e74b2-8125-42d1-b529-373d8aa7aa1e admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:22.959712 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-6e4e74b2-8125-42d1-b529-373d8aa7aa1e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:09:23.047722 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3aa01a-982a-4489-beb3-e1d831c163aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:23.050389 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3aa01a-982a-4489-beb3-e1d831c163aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:23.066995 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3aa01a-982a-4489-beb3-e1d831c163aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:23.067847 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3aa01a-982a-4489-beb3-e1d831c163aa admin admin] PATH_INFO: `/v3/users/7145c0ae0f3e4965907bcf282bdded82` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:23.070909 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3aa01a-982a-4489-beb3-e1d831c163aa admin admin] RBAC: Authorizing `identity:delete_user(user_id=7145c0ae0f3e4965907bcf282bdded82)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:23.074367 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3aa01a-982a-4489-beb3-e1d831c163aa admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:23.175005 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4e74b2-8125-42d1-b529-373d8aa7aa1e admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:23.221232 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab60294-9be2-4476-9223-b520777286c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:23.222223 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab60294-9be2-4476-9223-b520777286c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:23.222580 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab60294-9be2-4476-9223-b520777286c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:23.223317 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab60294-9be2-4476-9223-b520777286c3 admin admin] PATH_INFO: `/v3/projects/d8d41b2b380b48aa83e6a852a826615e` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:23.226062 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab60294-9be2-4476-9223-b520777286c3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d8d41b2b380b48aa83e6a852a826615e)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:23.229434 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab60294-9be2-4476-9223-b520777286c3 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:23.242071 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-5a3aa01a-982a-4489-beb3-e1d831c163aa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7145c0ae0f3e4965907bcf282bdded82'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:23.258203 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-5a3aa01a-982a-4489-beb3-e1d831c163aa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7145c0ae0f3e4965907bcf282bdded82'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:23.258203 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-5a3aa01a-982a-4489-beb3-e1d831c163aa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7145c0ae0f3e4965907bcf282bdded82'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:23.314231 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-5a3aa01a-982a-4489-beb3-e1d831c163aa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7145c0ae0f3e4965907bcf282bdded82'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:23.337120 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-5a3aa01a-982a-4489-beb3-e1d831c163aa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7145c0ae0f3e4965907bcf282bdded82'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:23.357339 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:09:23.359650 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 893/1765] 10.222.0.46 () {58 vars in 1223 bytes} [Thu May 26 15:09:22 2022] DELETE /identity/v3/users/7145c0ae0f3e4965907bcf282bdded82 => generated 0 bytes in 685 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:09:23.398133 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-6ab60294-9be2-4476-9223-b520777286c3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd8d41b2b380b48aa83e6a852a826615e', 'request_id': 'req-6ab60294-9be2-4476-9223-b520777286c3'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:23.407099 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-6ab60294-9be2-4476-9223-b520777286c3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd8d41b2b380b48aa83e6a852a826615e', 'request_id': 'req-6ab60294-9be2-4476-9223-b520777286c3'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:23.419632 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-6ab60294-9be2-4476-9223-b520777286c3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd8d41b2b380b48aa83e6a852a826615e', 'request_id': 'req-6ab60294-9be2-4476-9223-b520777286c3'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:23.459723 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-6ab60294-9be2-4476-9223-b520777286c3 admin admin] The token cache is being invalidate because project d8d41b2b380b48aa83e6a852a826615e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 15:09:23.460511 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-6ab60294-9be2-4476-9223-b520777286c3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:23.465979 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-6ab60294-9be2-4476-9223-b520777286c3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:23.471097 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 873/1766] 10.222.0.46 () {58 vars in 1232 bytes} [Thu May 26 15:09:22 2022] DELETE /identity/v3/projects/d8d41b2b380b48aa83e6a852a826615e => generated 0 bytes in 515 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:09:23.473006 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3aa01a-982a-4489-beb3-e1d831c163aa admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:23.475697 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-5a3aa01a-982a-4489-beb3-e1d831c163aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:09:23.632333 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:09:23.655531 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3aa01a-982a-4489-beb3-e1d831c163aa admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:23.665419 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58dd9b1a-da25-4a52-adbe-889d8f353114 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:23.666916 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dd9b1a-da25-4a52-adbe-889d8f353114 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:23.667349 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dd9b1a-da25-4a52-adbe-889d8f353114 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:23.668938 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58dd9b1a-da25-4a52-adbe-889d8f353114 admin admin] PATH_INFO: `/v3/users/b8213e6df38543db9bae0ca19d7f0aef` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:23.671677 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58dd9b1a-da25-4a52-adbe-889d8f353114 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b8213e6df38543db9bae0ca19d7f0aef)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:23.675325 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58dd9b1a-da25-4a52-adbe-889d8f353114 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:23.802448 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-58dd9b1a-da25-4a52-adbe-889d8f353114 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b8213e6df38543db9bae0ca19d7f0aef'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:23.819328 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-58dd9b1a-da25-4a52-adbe-889d8f353114 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b8213e6df38543db9bae0ca19d7f0aef'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:23.822210 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-58dd9b1a-da25-4a52-adbe-889d8f353114 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b8213e6df38543db9bae0ca19d7f0aef'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:23.843729 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab60294-9be2-4476-9223-b520777286c3 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:23.846730 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-6ab60294-9be2-4476-9223-b520777286c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:09:23.849890 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-58dd9b1a-da25-4a52-adbe-889d8f353114 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b8213e6df38543db9bae0ca19d7f0aef'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:23.866197 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-58dd9b1a-da25-4a52-adbe-889d8f353114 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b8213e6df38543db9bae0ca19d7f0aef'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:23.903164 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 894/1767] 10.222.0.46 () {58 vars in 1223 bytes} [Thu May 26 15:09:23 2022] DELETE /identity/v3/users/b8213e6df38543db9bae0ca19d7f0aef => generated 0 bytes in 428 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:09:24.097150 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab60294-9be2-4476-9223-b520777286c3 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:24.209496 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58dd9b1a-da25-4a52-adbe-889d8f353114 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:24.213032 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-58dd9b1a-da25-4a52-adbe-889d8f353114 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:09:24.243190 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d76d140-6a93-44d9-ac6e-1efbddf3de44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:24.246013 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d76d140-6a93-44d9-ac6e-1efbddf3de44 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:24.248104 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d76d140-6a93-44d9-ac6e-1efbddf3de44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:24.250943 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d76d140-6a93-44d9-ac6e-1efbddf3de44 admin admin] PATH_INFO: `/v3/projects/3a3c1e84727d40e28db1d10900ca0f40` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:24.255426 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d76d140-6a93-44d9-ac6e-1efbddf3de44 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3a3c1e84727d40e28db1d10900ca0f40)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:24.262535 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d76d140-6a93-44d9-ac6e-1efbddf3de44 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:24.290119 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58dd9b1a-da25-4a52-adbe-889d8f353114 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:24.302896 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a33a8e7d-c80f-47a7-9ba5-bfae04886b4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:24.304642 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33a8e7d-c80f-47a7-9ba5-bfae04886b4b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:24.305255 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33a8e7d-c80f-47a7-9ba5-bfae04886b4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:24.306155 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a33a8e7d-c80f-47a7-9ba5-bfae04886b4b admin admin] PATH_INFO: `/v3/projects/95ad0ab555bd4392926e926e53458a2f` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:24.309119 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33a8e7d-c80f-47a7-9ba5-bfae04886b4b admin admin] RBAC: Authorizing `identity:delete_project(project_id=95ad0ab555bd4392926e926e53458a2f)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:24.318497 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a33a8e7d-c80f-47a7-9ba5-bfae04886b4b admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:24.479485 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-a33a8e7d-c80f-47a7-9ba5-bfae04886b4b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '95ad0ab555bd4392926e926e53458a2f', 'request_id': 'req-a33a8e7d-c80f-47a7-9ba5-bfae04886b4b'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:24.479908 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-8d76d140-6a93-44d9-ac6e-1efbddf3de44 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3a3c1e84727d40e28db1d10900ca0f40', 'request_id': 'req-8d76d140-6a93-44d9-ac6e-1efbddf3de44'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:24.493470 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-8d76d140-6a93-44d9-ac6e-1efbddf3de44 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3a3c1e84727d40e28db1d10900ca0f40', 'request_id': 'req-8d76d140-6a93-44d9-ac6e-1efbddf3de44'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:24.495769 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-a33a8e7d-c80f-47a7-9ba5-bfae04886b4b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '95ad0ab555bd4392926e926e53458a2f', 'request_id': 'req-a33a8e7d-c80f-47a7-9ba5-bfae04886b4b'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:24.506329 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-8d76d140-6a93-44d9-ac6e-1efbddf3de44 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3a3c1e84727d40e28db1d10900ca0f40', 'request_id': 'req-8d76d140-6a93-44d9-ac6e-1efbddf3de44'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:24.513096 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:09:24.524503 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-a33a8e7d-c80f-47a7-9ba5-bfae04886b4b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '95ad0ab555bd4392926e926e53458a2f', 'request_id': 'req-a33a8e7d-c80f-47a7-9ba5-bfae04886b4b'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:24.534934 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-8d76d140-6a93-44d9-ac6e-1efbddf3de44 admin admin] The token cache is being invalidate because project 3a3c1e84727d40e28db1d10900ca0f40 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 15:09:24.538129 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-8d76d140-6a93-44d9-ac6e-1efbddf3de44 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:24.542120 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-8d76d140-6a93-44d9-ac6e-1efbddf3de44 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:24.553406 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 874/1768] 10.222.0.46 () {58 vars in 1232 bytes} [Thu May 26 15:09:23 2022] DELETE /identity/v3/projects/3a3c1e84727d40e28db1d10900ca0f40 => generated 0 bytes in 707 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:09:24.555046 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5893c35-5537-4983-aab3-dca5339ef395 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:24.555524 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5893c35-5537-4983-aab3-dca5339ef395 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:24.556480 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5893c35-5537-4983-aab3-dca5339ef395 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:24.559293 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 875/1769] 10.222.0.46 () {54 vars in 912 bytes} [Thu May 26 15:09:24 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 15:09:24.564742 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4427b719-409b-438e-8c80-b5730a24be90 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:24.566847 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4427b719-409b-438e-8c80-b5730a24be90 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:24.568315 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4427b719-409b-438e-8c80-b5730a24be90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:24.568667 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-a33a8e7d-c80f-47a7-9ba5-bfae04886b4b admin admin] The token cache is being invalidate because project 95ad0ab555bd4392926e926e53458a2f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 15:09:24.571282 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-a33a8e7d-c80f-47a7-9ba5-bfae04886b4b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:24.572676 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-a33a8e7d-c80f-47a7-9ba5-bfae04886b4b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:24.580750 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 895/1770] 10.222.0.46 () {58 vars in 1232 bytes} [Thu May 26 15:09:24 2022] DELETE /identity/v3/projects/95ad0ab555bd4392926e926e53458a2f => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:09:24.585478 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6915595-653f-4d7a-8383-cd5d80ddcbe4 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:24.587224 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6915595-653f-4d7a-8383-cd5d80ddcbe4 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:24.588139 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6915595-653f-4d7a-8383-cd5d80ddcbe4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:24.625755 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-4427b719-409b-438e-8c80-b5730a24be90 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:09:24.625755 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-4427b719-409b-438e-8c80-b5730a24be90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:09:24.634849 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-c6915595-653f-4d7a-8383-cd5d80ddcbe4 None None] MFA Rules not processed for user `0156a2f6f3794f479e76b85bdbbe5c26`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:09:24.638743 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-c6915595-653f-4d7a-8383-cd5d80ddcbe4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:09:24.703680 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:09:24.716959 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 896/1771] 10.222.0.46 () {58 vars in 1009 bytes} [Thu May 26 15:09:24 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:09:24.719839 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44d8f0f-19a9-4b89-a0c9-40a080a00836 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:24.720546 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44d8f0f-19a9-4b89-a0c9-40a080a00836 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:24.721074 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44d8f0f-19a9-4b89-a0c9-40a080a00836 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:24.740617 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 876/1772] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:09:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:09:24.741588 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4427b719-409b-438e-8c80-b5730a24be90 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:24.743018 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-4427b719-409b-438e-8c80-b5730a24be90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:09:24.747987 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-a44d8f0f-19a9-4b89-a0c9-40a080a00836 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:09:24.750324 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-a44d8f0f-19a9-4b89-a0c9-40a080a00836 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:09:24.758043 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4427b719-409b-438e-8c80-b5730a24be90 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:24.763624 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea1da5a-cb6b-4b88-a002-434d814d345a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:24.763956 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 897/1773] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:09:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:09:24.768772 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb53197-f98e-4453-aa7a-4f7677f8b409 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:24.769578 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb53197-f98e-4453-aa7a-4f7677f8b409 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:24.770679 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb53197-f98e-4453-aa7a-4f7677f8b409 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:24.773298 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea1da5a-cb6b-4b88-a002-434d814d345a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:24.773693 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea1da5a-cb6b-4b88-a002-434d814d345a admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:24.774067 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea1da5a-cb6b-4b88-a002-434d814d345a admin admin] PATH_INFO: `/v3/projects/b2135fbd21c845c0ab71d02d5be2d3d4` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:24.776275 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea1da5a-cb6b-4b88-a002-434d814d345a admin admin] RBAC: Authorizing `identity:delete_project(project_id=b2135fbd21c845c0ab71d02d5be2d3d4)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:24.778363 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea1da5a-cb6b-4b88-a002-434d814d345a admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:24.821084 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-0eb53197-f98e-4453-aa7a-4f7677f8b409 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:09:24.823294 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-0eb53197-f98e-4453-aa7a-4f7677f8b409 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:09:24.839846 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 898/1774] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:09:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:09:24.839846 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785e70be-c4a8-4b98-8830-7c201114a85b None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:24.839846 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785e70be-c4a8-4b98-8830-7c201114a85b None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:24.840640 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-785e70be-c4a8-4b98-8830-7c201114a85b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:24.871191 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-785e70be-c4a8-4b98-8830-7c201114a85b None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:09:24.872954 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-785e70be-c4a8-4b98-8830-7c201114a85b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:09:24.887461 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 899/1775] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:09:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:09:24.888583 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785e70be-c4a8-4b98-8830-7c201114a85b None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:24.890081 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-785e70be-c4a8-4b98-8830-7c201114a85b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:09:24.911225 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-785e70be-c4a8-4b98-8830-7c201114a85b None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:24.912310 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-1ea1da5a-cb6b-4b88-a002-434d814d345a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b2135fbd21c845c0ab71d02d5be2d3d4', 'request_id': 'req-1ea1da5a-cb6b-4b88-a002-434d814d345a'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:24.921529 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-1ea1da5a-cb6b-4b88-a002-434d814d345a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b2135fbd21c845c0ab71d02d5be2d3d4', 'request_id': 'req-1ea1da5a-cb6b-4b88-a002-434d814d345a'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:24.935300 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-1ea1da5a-cb6b-4b88-a002-434d814d345a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b2135fbd21c845c0ab71d02d5be2d3d4', 'request_id': 'req-1ea1da5a-cb6b-4b88-a002-434d814d345a'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:24.969794 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-1ea1da5a-cb6b-4b88-a002-434d814d345a admin admin] The token cache is being invalidate because project b2135fbd21c845c0ab71d02d5be2d3d4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 15:09:24.969981 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-1ea1da5a-cb6b-4b88-a002-434d814d345a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:24.970719 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-1ea1da5a-cb6b-4b88-a002-434d814d345a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:24.972513 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 877/1776] 10.222.0.46 () {58 vars in 1232 bytes} [Thu May 26 15:09:24 2022] DELETE /identity/v3/projects/b2135fbd21c845c0ab71d02d5be2d3d4 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:09:24.974124 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f32abbb-830e-4451-91ab-85cc3bb4b59f None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:24.974415 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f32abbb-830e-4451-91ab-85cc3bb4b59f None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:24.974692 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f32abbb-830e-4451-91ab-85cc3bb4b59f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:24.994382 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c903bc8-7bfd-4602-afde-aed4d17f436c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0156a2f6f3794f479e76b85bdbbe5c26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:24.995103 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c903bc8-7bfd-4602-afde-aed4d17f436c service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:24.995391 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c903bc8-7bfd-4602-afde-aed4d17f436c service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:24.995713 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c903bc8-7bfd-4602-afde-aed4d17f436c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:24.996749 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-6f32abbb-830e-4451-91ab-85cc3bb4b59f None None] MFA Rules not processed for user `2ce5b5d5a12743bc80a4b191c24ff52f`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:09:24.997890 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-7c903bc8-7bfd-4602-afde-aed4d17f436c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:09:24.998324 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-6f32abbb-830e-4451-91ab-85cc3bb4b59f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:09:25.009999 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c903bc8-7bfd-4602-afde-aed4d17f436c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:25.011630 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c903bc8-7bfd-4602-afde-aed4d17f436c service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:25.047698 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 900/1777] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:09:24 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 15:09:25.049848 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c903bc8-7bfd-4602-afde-aed4d17f436c service neutron] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:25.051625 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-7c903bc8-7bfd-4602-afde-aed4d17f436c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:09:25.089116 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 878/1778] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:09:24 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:09:25.094620 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f32abbb-830e-4451-91ab-85cc3bb4b59f None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:25.097719 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-6f32abbb-830e-4451-91ab-85cc3bb4b59f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:09:25.142361 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:09:25.233126 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c903bc8-7bfd-4602-afde-aed4d17f436c service neutron] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:25.234291 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f32abbb-830e-4451-91ab-85cc3bb4b59f None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:25.240818 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968271f6-c0aa-4b31-95c5-5a36b57ab031 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:25.241785 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968271f6-c0aa-4b31-95c5-5a36b57ab031 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:25.242251 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968271f6-c0aa-4b31-95c5-5a36b57ab031 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:25.242676 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968271f6-c0aa-4b31-95c5-5a36b57ab031 admin admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:25.244739 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968271f6-c0aa-4b31-95c5-5a36b57ab031 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 15:09:25.245742 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968271f6-c0aa-4b31-95c5-5a36b57ab031 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:25.247946 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968271f6-c0aa-4b31-95c5-5a36b57ab031 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:25.251855 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd77a50-df1c-42a3-86c3-a4d10294bd37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:25.253056 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd77a50-df1c-42a3-86c3-a4d10294bd37 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:25.254069 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd77a50-df1c-42a3-86c3-a4d10294bd37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:25.254069 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd77a50-df1c-42a3-86c3-a4d10294bd37 admin admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:25.254500 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd77a50-df1c-42a3-86c3-a4d10294bd37 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 15:09:25.254820 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd77a50-df1c-42a3-86c3-a4d10294bd37 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:25.256513 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd77a50-df1c-42a3-86c3-a4d10294bd37 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:25.270679 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 879/1779] 10.222.0.46 () {58 vars in 1152 bytes} [Thu May 26 15:09:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 15:09:25.270679 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7591f921-15f4-4cee-8637-cc6a90d18082 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:25.270679 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7591f921-15f4-4cee-8637-cc6a90d18082 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:25.270679 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7591f921-15f4-4cee-8637-cc6a90d18082 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:25.298309 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 901/1780] 10.222.0.46 () {58 vars in 1152 bytes} [Thu May 26 15:09:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 15:09:25.299933 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd77a50-df1c-42a3-86c3-a4d10294bd37 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:25.303547 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-7591f921-15f4-4cee-8637-cc6a90d18082 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:09:25.305412 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-7591f921-15f4-4cee-8637-cc6a90d18082 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:09:25.314061 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-bfd77a50-df1c-42a3-86c3-a4d10294bd37 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:09:25.324573 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 880/1781] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:09:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:09:25.342309 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7591f921-15f4-4cee-8637-cc6a90d18082 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:25.351528 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd77a50-df1c-42a3-86c3-a4d10294bd37 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:25.366566 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6c12f5-dd6f-497d-8a76-5685b335fe4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:25.372001 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6c12f5-dd6f-497d-8a76-5685b335fe4d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:25.372001 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6c12f5-dd6f-497d-8a76-5685b335fe4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:25.372001 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6c12f5-dd6f-497d-8a76-5685b335fe4d admin admin] PATH_INFO: `/v3/users/8e3e2483a9f94a759da1bb8fab329987` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:25.372582 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7591f921-15f4-4cee-8637-cc6a90d18082 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:25.372729 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6c12f5-dd6f-497d-8a76-5685b335fe4d admin admin] RBAC: Authorizing `identity:delete_user(user_id=8e3e2483a9f94a759da1bb8fab329987)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:25.375137 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6c12f5-dd6f-497d-8a76-5685b335fe4d admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:25.424788 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2de3f3-f4fa-434c-b423-78b42fdec829 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:25.427030 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2de3f3-f4fa-434c-b423-78b42fdec829 admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:25.427521 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2de3f3-f4fa-434c-b423-78b42fdec829 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:25.429022 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2de3f3-f4fa-434c-b423-78b42fdec829 admin admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:25.431187 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2de3f3-f4fa-434c-b423-78b42fdec829 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:25.435968 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec2de3f3-f4fa-434c-b423-78b42fdec829 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:25.491654 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 881/1782] 10.222.0.46 () {60 vars in 1152 bytes} [Thu May 26 15:09:25 2022] POST /identity/v3/projects => generated 393 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 15:09:25.492886 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2de3f3-f4fa-434c-b423-78b42fdec829 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:25.493039 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-fa6c12f5-dd6f-497d-8a76-5685b335fe4d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8e3e2483a9f94a759da1bb8fab329987'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:25.509358 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-fa6c12f5-dd6f-497d-8a76-5685b335fe4d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8e3e2483a9f94a759da1bb8fab329987'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:25.510545 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-fa6c12f5-dd6f-497d-8a76-5685b335fe4d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8e3e2483a9f94a759da1bb8fab329987'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:25.537953 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:09:25.539971 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-fa6c12f5-dd6f-497d-8a76-5685b335fe4d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8e3e2483a9f94a759da1bb8fab329987'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:25.554737 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-fa6c12f5-dd6f-497d-8a76-5685b335fe4d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8e3e2483a9f94a759da1bb8fab329987'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:25.571340 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 902/1783] 10.222.0.46 () {58 vars in 1223 bytes} [Thu May 26 15:09:25 2022] DELETE /identity/v3/users/8e3e2483a9f94a759da1bb8fab329987 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:09:25.572986 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558b84d8-12ae-480b-ab65-9901b42b076c None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:25.573485 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558b84d8-12ae-480b-ab65-9901b42b076c None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:25.573902 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558b84d8-12ae-480b-ab65-9901b42b076c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:25.618423 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-558b84d8-12ae-480b-ab65-9901b42b076c None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:09:25.622826 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-558b84d8-12ae-480b-ab65-9901b42b076c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:09:25.657709 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2de3f3-f4fa-434c-b423-78b42fdec829 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:25.748229 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 903/1784] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:09:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:09:25.750581 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558b84d8-12ae-480b-ab65-9901b42b076c None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:25.752750 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-558b84d8-12ae-480b-ab65-9901b42b076c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:09:25.767837 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558b84d8-12ae-480b-ab65-9901b42b076c None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:25.774457 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d451dfd-2645-47c8-afd4-1ef0bed44108 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:25.775213 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d451dfd-2645-47c8-afd4-1ef0bed44108 admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:25.775506 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d451dfd-2645-47c8-afd4-1ef0bed44108 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:25.775877 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d451dfd-2645-47c8-afd4-1ef0bed44108 admin admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:25.777799 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d451dfd-2645-47c8-afd4-1ef0bed44108 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:25.779612 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d451dfd-2645-47c8-afd4-1ef0bed44108 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:25.780152 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637ba7e2-57ad-4d5e-ae7d-eb656c79cc81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:25.781217 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637ba7e2-57ad-4d5e-ae7d-eb656c79cc81 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:25.782108 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637ba7e2-57ad-4d5e-ae7d-eb656c79cc81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:25.782606 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637ba7e2-57ad-4d5e-ae7d-eb656c79cc81 admin admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:25.783965 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637ba7e2-57ad-4d5e-ae7d-eb656c79cc81 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:25.787209 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637ba7e2-57ad-4d5e-ae7d-eb656c79cc81 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:25.825188 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 882/1785] 10.222.0.46 () {60 vars in 1152 bytes} [Thu May 26 15:09:25 2022] POST /identity/v3/projects => generated 373 bytes in 333 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 15:09:25.827381 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc8ee44-0f52-467f-90bf-dd15e76c4be8 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:25.827847 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc8ee44-0f52-467f-90bf-dd15e76c4be8 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:25.828576 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc8ee44-0f52-467f-90bf-dd15e76c4be8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:25.832885 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 904/1786] 10.222.0.46 () {60 vars in 1143 bytes} [Thu May 26 15:09:25 2022] POST /identity/v3/users => generated 297 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 15:09:25.833809 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637ba7e2-57ad-4d5e-ae7d-eb656c79cc81 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:25.845425 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637ba7e2-57ad-4d5e-ae7d-eb656c79cc81 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:25.855086 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-9dc8ee44-0f52-467f-90bf-dd15e76c4be8 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:09:25.857175 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-9dc8ee44-0f52-467f-90bf-dd15e76c4be8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:09:25.881378 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:09:25.913403 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0281e20-d2ea-45ab-8186-f57c0655cc72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:25.914310 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0281e20-d2ea-45ab-8186-f57c0655cc72 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:25.914620 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0281e20-d2ea-45ab-8186-f57c0655cc72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:25.914905 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0281e20-d2ea-45ab-8186-f57c0655cc72 admin admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:25.915664 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0281e20-d2ea-45ab-8186-f57c0655cc72 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 15:09:25.915981 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0281e20-d2ea-45ab-8186-f57c0655cc72 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:25.917668 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0281e20-d2ea-45ab-8186-f57c0655cc72 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:25.945513 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 905/1787] 10.222.0.46 () {58 vars in 1152 bytes} [Thu May 26 15:09:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 15:09:25.950085 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0281e20-d2ea-45ab-8186-f57c0655cc72 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:25.954325 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-b0281e20-d2ea-45ab-8186-f57c0655cc72 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:09:25.959675 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 883/1788] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:09:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:09:25.961250 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc8ee44-0f52-467f-90bf-dd15e76c4be8 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:25.973063 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0281e20-d2ea-45ab-8186-f57c0655cc72 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:25.975687 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc8ee44-0f52-467f-90bf-dd15e76c4be8 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:25.981177 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cf5bfb-55ce-4316-9c51-38c94e8862ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:25.982651 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cf5bfb-55ce-4316-9c51-38c94e8862ee admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:25.983040 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cf5bfb-55ce-4316-9c51-38c94e8862ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:25.983471 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cf5bfb-55ce-4316-9c51-38c94e8862ee admin admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:25.983841 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7a510c-ecc0-4938-9ac6-9714eeda8000 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:25.984581 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7a510c-ecc0-4938-9ac6-9714eeda8000 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:25.984852 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8cf5bfb-55ce-4316-9c51-38c94e8862ee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:25.984970 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7a510c-ecc0-4938-9ac6-9714eeda8000 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:25.985131 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7a510c-ecc0-4938-9ac6-9714eeda8000 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:25.985973 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7a510c-ecc0-4938-9ac6-9714eeda8000 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:25.987547 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8cf5bfb-55ce-4316-9c51-38c94e8862ee admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:25.987730 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7a510c-ecc0-4938-9ac6-9714eeda8000 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:26.005258 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 884/1789] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 15:09:25 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 15:09:26.006278 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7a510c-ecc0-4938-9ac6-9714eeda8000 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:26.020803 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7a510c-ecc0-4938-9ac6-9714eeda8000 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:26.027447 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b851812-85b6-4585-95b7-dda302396f48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:26.028321 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b851812-85b6-4585-95b7-dda302396f48 admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:26.028705 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b851812-85b6-4585-95b7-dda302396f48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:26.029092 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b851812-85b6-4585-95b7-dda302396f48 admin admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:26.031240 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b851812-85b6-4585-95b7-dda302396f48 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:26.033683 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b851812-85b6-4585-95b7-dda302396f48 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:26.047847 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 906/1790] 10.222.0.46 () {60 vars in 1143 bytes} [Thu May 26 15:09:25 2022] POST /identity/v3/users => generated 287 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 15:09:26.047847 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cf5bfb-55ce-4316-9c51-38c94e8862ee admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:26.050291 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-e8cf5bfb-55ce-4316-9c51-38c94e8862ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:09:26.093670 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 885/1791] 10.222.0.46 () {60 vars in 1152 bytes} [Thu May 26 15:09:26 2022] POST /identity/v3/projects => generated 363 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 15:09:26.094846 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b851812-85b6-4585-95b7-dda302396f48 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:26.141732 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:09:26.153976 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cf5bfb-55ce-4316-9c51-38c94e8862ee admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:26.191506 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b851812-85b6-4585-95b7-dda302396f48 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:26.196981 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0d3279-738f-481c-b187-e618e6c5c44f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:26.197678 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0d3279-738f-481c-b187-e618e6c5c44f admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:26.198178 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0d3279-738f-481c-b187-e618e6c5c44f admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:26.198474 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0d3279-738f-481c-b187-e618e6c5c44f admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:26.199347 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0d3279-738f-481c-b187-e618e6c5c44f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:26.201559 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0d3279-738f-481c-b187-e618e6c5c44f admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:26.222379 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 886/1792] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 15:09:26 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 15:09:26.223402 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0d3279-738f-481c-b187-e618e6c5c44f admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:26.259857 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0d3279-738f-481c-b187-e618e6c5c44f admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:26.270702 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9636bfb5-e78f-4067-bc74-f14d59b1b998 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:26.271363 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9636bfb5-e78f-4067-bc74-f14d59b1b998 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:26.271783 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9636bfb5-e78f-4067-bc74-f14d59b1b998 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:26.272136 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9636bfb5-e78f-4067-bc74-f14d59b1b998 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:26.273811 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9636bfb5-e78f-4067-bc74-f14d59b1b998 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:26.277328 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9636bfb5-e78f-4067-bc74-f14d59b1b998 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:26.297595 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 887/1793] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 15:09:26 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 15:09:26.298746 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9636bfb5-e78f-4067-bc74-f14d59b1b998 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:26.308470 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f1753a-84ad-4cdc-a30c-0a7f23ba58a4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0156a2f6f3794f479e76b85bdbbe5c26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:26.310425 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f1753a-84ad-4cdc-a30c-0a7f23ba58a4 service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:26.310425 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f1753a-84ad-4cdc-a30c-0a7f23ba58a4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:26.310661 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f1753a-84ad-4cdc-a30c-0a7f23ba58a4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:26.320879 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9636bfb5-e78f-4067-bc74-f14d59b1b998 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:26.329841 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f1753a-84ad-4cdc-a30c-0a7f23ba58a4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:26.331480 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687c9b85-7c95-4e2e-a816-b931388d31b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:26.332625 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687c9b85-7c95-4e2e-a816-b931388d31b0 admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:26.333067 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f1753a-84ad-4cdc-a30c-0a7f23ba58a4 service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:26.333184 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687c9b85-7c95-4e2e-a816-b931388d31b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:26.333514 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687c9b85-7c95-4e2e-a816-b931388d31b0 admin admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:26.335265 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687c9b85-7c95-4e2e-a816-b931388d31b0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:26.338452 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687c9b85-7c95-4e2e-a816-b931388d31b0 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:26.354394 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 907/1794] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:09:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 15:09:26.357474 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f1753a-84ad-4cdc-a30c-0a7f23ba58a4 service neutron] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:26.376249 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f1753a-84ad-4cdc-a30c-0a7f23ba58a4 service neutron] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:26.384380 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa513a24-c542-46a9-8382-93ab32dc4668 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:26.385754 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa513a24-c542-46a9-8382-93ab32dc4668 admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:26.386244 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa513a24-c542-46a9-8382-93ab32dc4668 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:26.387021 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa513a24-c542-46a9-8382-93ab32dc4668 admin admin] PATH_INFO: `/v3/projects/27eba87c67f1479196cfe6b904db154f/users/fe57a7ef56fb4437b4c981b312ab8c7d/roles/d4088d1845a24fbd9bd8975ce0b1d3d0` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:26.391169 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 888/1795] 10.222.0.46 () {60 vars in 1143 bytes} [Thu May 26 15:09:26 2022] POST /identity/v3/users => generated 282 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 15:09:26.392639 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687c9b85-7c95-4e2e-a816-b931388d31b0 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:26.410486 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa513a24-c542-46a9-8382-93ab32dc4668 admin admin] RBAC: Authorizing `identity:create_grant(project_id=27eba87c67f1479196cfe6b904db154f, user_id=fe57a7ef56fb4437b4c981b312ab8c7d, role_id=d4088d1845a24fbd9bd8975ce0b1d3d0)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:26.414467 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa513a24-c542-46a9-8382-93ab32dc4668 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:26.420078 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687c9b85-7c95-4e2e-a816-b931388d31b0 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:26.435619 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11014648-73ea-45b9-83f0-f591f37859cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:26.436493 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 908/1796] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 15:09:26 2022] PUT /identity/v3/projects/27eba87c67f1479196cfe6b904db154f/users/fe57a7ef56fb4437b4c981b312ab8c7d/roles/d4088d1845a24fbd9bd8975ce0b1d3d0 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:09:26.437157 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11014648-73ea-45b9-83f0-f591f37859cf admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:26.437644 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11014648-73ea-45b9-83f0-f591f37859cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:26.437963 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa513a24-c542-46a9-8382-93ab32dc4668 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:26.438355 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11014648-73ea-45b9-83f0-f591f37859cf admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:26.440458 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11014648-73ea-45b9-83f0-f591f37859cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:26.442577 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11014648-73ea-45b9-83f0-f591f37859cf admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:26.478166 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 889/1797] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 15:09:26 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 15:09:26.482367 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e762726-64fa-4c7c-82da-6d3b69bb634f None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:26.482801 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e762726-64fa-4c7c-82da-6d3b69bb634f None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:26.483494 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e762726-64fa-4c7c-82da-6d3b69bb634f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:26.546465 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-2e762726-64fa-4c7c-82da-6d3b69bb634f None None] MFA Rules not processed for user `fe57a7ef56fb4437b4c981b312ab8c7d`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:09:26.547729 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-2e762726-64fa-4c7c-82da-6d3b69bb634f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:09:26.582410 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa513a24-c542-46a9-8382-93ab32dc4668 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:26.587903 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c8cb05-00d1-4224-9282-a159d7eaa4ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:26.588645 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c8cb05-00d1-4224-9282-a159d7eaa4ea admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:26.588941 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c8cb05-00d1-4224-9282-a159d7eaa4ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:26.589332 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c8cb05-00d1-4224-9282-a159d7eaa4ea admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:26.589994 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c8cb05-00d1-4224-9282-a159d7eaa4ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:26.592217 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83c8cb05-00d1-4224-9282-a159d7eaa4ea admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:26.629934 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 909/1798] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 15:09:26 2022] GET /identity/v3/roles => generated 1420 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 15:09:26.630932 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c8cb05-00d1-4224-9282-a159d7eaa4ea admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:26.649866 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83c8cb05-00d1-4224-9282-a159d7eaa4ea admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:26.667803 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803241e7-5eae-47bd-be01-49077f792c9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:26.669785 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803241e7-5eae-47bd-be01-49077f792c9b admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:26.670657 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803241e7-5eae-47bd-be01-49077f792c9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:26.672220 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803241e7-5eae-47bd-be01-49077f792c9b admin admin] PATH_INFO: `/v3/projects/edc7098ee470478e8c24a8affc9d5161/users/c431cf6affab4ba7a916ff8b829cd7b7/roles/d4088d1845a24fbd9bd8975ce0b1d3d0` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:26.715202 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803241e7-5eae-47bd-be01-49077f792c9b admin admin] RBAC: Authorizing `identity:create_grant(project_id=edc7098ee470478e8c24a8affc9d5161, user_id=c431cf6affab4ba7a916ff8b829cd7b7, role_id=d4088d1845a24fbd9bd8975ce0b1d3d0)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:26.718758 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803241e7-5eae-47bd-be01-49077f792c9b admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:26.738352 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:09:26.774809 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 910/1799] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 15:09:26 2022] PUT /identity/v3/projects/edc7098ee470478e8c24a8affc9d5161/users/c431cf6affab4ba7a916ff8b829cd7b7/roles/d4088d1845a24fbd9bd8975ce0b1d3d0 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:09:26.799318 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803241e7-5eae-47bd-be01-49077f792c9b admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:26.852547 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 890/1800] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:09:26 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 374 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:09:26.857132 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca47a38-6102-48b7-b3ed-1e59b718b9b5 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:26.857132 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca47a38-6102-48b7-b3ed-1e59b718b9b5 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:26.858080 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca47a38-6102-48b7-b3ed-1e59b718b9b5 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:26.864118 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 891/1801] 10.222.0.46 () {54 vars in 921 bytes} [Thu May 26 15:09:26 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 15:09:26.870308 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d42006-4fc0-4332-9c35-4c073b10d47e None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:26.870520 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d42006-4fc0-4332-9c35-4c073b10d47e None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:26.870824 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d42006-4fc0-4332-9c35-4c073b10d47e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:26.971003 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:09:26.971740 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-d1d42006-4fc0-4332-9c35-4c073b10d47e None None] MFA Rules not processed for user `c431cf6affab4ba7a916ff8b829cd7b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:09:26.972559 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-d1d42006-4fc0-4332-9c35-4c073b10d47e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:09:27.008853 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803241e7-5eae-47bd-be01-49077f792c9b admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:27.020529 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336108e6-2d4c-4059-8cf6-e1833d2c2f46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:27.021217 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336108e6-2d4c-4059-8cf6-e1833d2c2f46 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:27.021526 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336108e6-2d4c-4059-8cf6-e1833d2c2f46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:27.022347 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336108e6-2d4c-4059-8cf6-e1833d2c2f46 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:27.022655 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336108e6-2d4c-4059-8cf6-e1833d2c2f46 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:27.024320 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336108e6-2d4c-4059-8cf6-e1833d2c2f46 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:27.039647 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 911/1802] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 15:09:26 2022] GET /identity/v3/roles => generated 1420 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 15:09:27.041041 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336108e6-2d4c-4059-8cf6-e1833d2c2f46 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:27.060881 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336108e6-2d4c-4059-8cf6-e1833d2c2f46 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:27.069990 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6180700d-2ffa-42c1-b915-f7016cefc24f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:27.072165 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6180700d-2ffa-42c1-b915-f7016cefc24f admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:27.074707 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6180700d-2ffa-42c1-b915-f7016cefc24f admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:27.075375 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6180700d-2ffa-42c1-b915-f7016cefc24f admin admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:27.078919 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6180700d-2ffa-42c1-b915-f7016cefc24f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:27.081270 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6180700d-2ffa-42c1-b915-f7016cefc24f admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:27.132551 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 912/1803] 10.222.0.46 () {60 vars in 1152 bytes} [Thu May 26 15:09:27 2022] POST /identity/v3/projects => generated 389 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 15:09:27.133917 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c4c506-396c-49bb-b6e9-a6d7b5db84bc None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:27.134274 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c4c506-396c-49bb-b6e9-a6d7b5db84bc None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:27.134618 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00c4c506-396c-49bb-b6e9-a6d7b5db84bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:27.155399 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-00c4c506-396c-49bb-b6e9-a6d7b5db84bc None None] MFA Rules not processed for user `d6639353c70449d1b3d10cd6487c3b00`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:09:27.156762 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-00c4c506-396c-49bb-b6e9-a6d7b5db84bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:09:27.189833 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 892/1804] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:09:26 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 325 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:09:27.190788 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d42006-4fc0-4332-9c35-4c073b10d47e None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:27.219235 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d42006-4fc0-4332-9c35-4c073b10d47e None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:27.281592 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 913/1805] 10.222.0.46 () {58 vars in 1018 bytes} [Thu May 26 15:09:27 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:09:27.283102 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c4c506-396c-49bb-b6e9-a6d7b5db84bc None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:27.380726 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d6b106-331f-48db-b41d-1fbc785fb047 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:27.381714 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d6b106-331f-48db-b41d-1fbc785fb047 admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:27.382000 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d6b106-331f-48db-b41d-1fbc785fb047 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:27.382278 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d6b106-331f-48db-b41d-1fbc785fb047 admin admin] PATH_INFO: `/v3/projects/76b9a456767c4aa99d73aee5a20a6172/users/8afca1c17da04182ad09bf43a8a4284a/roles/d4088d1845a24fbd9bd8975ce0b1d3d0` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:27.428077 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d6b106-331f-48db-b41d-1fbc785fb047 admin admin] RBAC: Authorizing `identity:create_grant(project_id=76b9a456767c4aa99d73aee5a20a6172, user_id=8afca1c17da04182ad09bf43a8a4284a, role_id=d4088d1845a24fbd9bd8975ce0b1d3d0)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:27.438926 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03d6b106-331f-48db-b41d-1fbc785fb047 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:27.455738 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00c4c506-396c-49bb-b6e9-a6d7b5db84bc None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:27.475363 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 893/1806] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 15:09:27 2022] PUT /identity/v3/projects/76b9a456767c4aa99d73aee5a20a6172/users/8afca1c17da04182ad09bf43a8a4284a/roles/d4088d1845a24fbd9bd8975ce0b1d3d0 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:09:27.478527 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addf0d94-45ce-4289-851c-0ec40195d0f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:27.479793 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addf0d94-45ce-4289-851c-0ec40195d0f3 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:27.480270 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addf0d94-45ce-4289-851c-0ec40195d0f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:27.481971 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addf0d94-45ce-4289-851c-0ec40195d0f3 admin admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:27.482642 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addf0d94-45ce-4289-851c-0ec40195d0f3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:27.484424 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d6b106-331f-48db-b41d-1fbc785fb047 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:27.489848 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addf0d94-45ce-4289-851c-0ec40195d0f3 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:27.491555 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-03d6b106-331f-48db-b41d-1fbc785fb047 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:09:27.537788 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 914/1807] 10.222.0.46 () {60 vars in 1143 bytes} [Thu May 26 15:09:27 2022] POST /identity/v3/users => generated 301 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 15:09:27.537788 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addf0d94-45ce-4289-851c-0ec40195d0f3 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:27.542008 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-addf0d94-45ce-4289-851c-0ec40195d0f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:09:27.560821 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addf0d94-45ce-4289-851c-0ec40195d0f3 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:27.610583 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:09:27.623469 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d6b106-331f-48db-b41d-1fbc785fb047 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:27.636425 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a922caf-d565-4d6c-8406-938f2f16a3ec service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5b5b35614a547d783e6e77ddb8c1c03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:27.638333 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a922caf-d565-4d6c-8406-938f2f16a3ec service cinder] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:27.638981 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a922caf-d565-4d6c-8406-938f2f16a3ec service cinder] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:27.639839 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a922caf-d565-4d6c-8406-938f2f16a3ec service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:27.666261 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:09:27.705489 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a6ca3c-9f82-48e4-ad71-8ed24b5b2f8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:27.705489 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a6ca3c-9f82-48e4-ad71-8ed24b5b2f8a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:27.705489 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a6ca3c-9f82-48e4-ad71-8ed24b5b2f8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:27.705489 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a6ca3c-9f82-48e4-ad71-8ed24b5b2f8a admin admin] PATH_INFO: `/v3/projects/ed634dcb373941e2b612f1402f5aedb2` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:27.707038 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a6ca3c-9f82-48e4-ad71-8ed24b5b2f8a admin admin] RBAC: Authorizing `identity:delete_project(project_id=ed634dcb373941e2b612f1402f5aedb2)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:27.709857 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a6ca3c-9f82-48e4-ad71-8ed24b5b2f8a admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:27.820023 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-87a6ca3c-9f82-48e4-ad71-8ed24b5b2f8a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ed634dcb373941e2b612f1402f5aedb2', 'request_id': 'req-87a6ca3c-9f82-48e4-ad71-8ed24b5b2f8a'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:27.836351 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a922caf-d565-4d6c-8406-938f2f16a3ec service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:27.843534 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a922caf-d565-4d6c-8406-938f2f16a3ec service cinder] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:27.844424 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-87a6ca3c-9f82-48e4-ad71-8ed24b5b2f8a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ed634dcb373941e2b612f1402f5aedb2', 'request_id': 'req-87a6ca3c-9f82-48e4-ad71-8ed24b5b2f8a'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:27.862867 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-87a6ca3c-9f82-48e4-ad71-8ed24b5b2f8a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ed634dcb373941e2b612f1402f5aedb2', 'request_id': 'req-87a6ca3c-9f82-48e4-ad71-8ed24b5b2f8a'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:27.885840 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 894/1808] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:09:27 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 15:09:27.890183 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5afbcb-ab92-4dd1-a41c-468ff448cf61 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:27.890870 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5afbcb-ab92-4dd1-a41c-468ff448cf61 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:27.893613 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb5afbcb-ab92-4dd1-a41c-468ff448cf61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:27.897360 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-87a6ca3c-9f82-48e4-ad71-8ed24b5b2f8a admin admin] The token cache is being invalidate because project ed634dcb373941e2b612f1402f5aedb2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 15:09:27.897748 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-87a6ca3c-9f82-48e4-ad71-8ed24b5b2f8a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:27.902460 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-87a6ca3c-9f82-48e4-ad71-8ed24b5b2f8a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:27.910796 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 915/1809] 10.222.0.46 () {58 vars in 1232 bytes} [Thu May 26 15:09:27 2022] DELETE /identity/v3/projects/ed634dcb373941e2b612f1402f5aedb2 => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:09:27.910796 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a6ca3c-9f82-48e4-ad71-8ed24b5b2f8a admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:27.913809 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-87a6ca3c-9f82-48e4-ad71-8ed24b5b2f8a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:09:27.991768 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-bb5afbcb-ab92-4dd1-a41c-468ff448cf61 None None] MFA Rules not processed for user `8afca1c17da04182ad09bf43a8a4284a`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:09:27.995188 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-bb5afbcb-ab92-4dd1-a41c-468ff448cf61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:09:28.145936 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a6ca3c-9f82-48e4-ad71-8ed24b5b2f8a admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:28.151496 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 895/1810] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:09:27 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:09:28.152515 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df55cd8e-1df5-48cf-8a3e-bc3108b37c4f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6639353c70449d1b3d10cd6487c3b00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:28.153148 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5afbcb-ab92-4dd1-a41c-468ff448cf61 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:28.154452 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df55cd8e-1df5-48cf-8a3e-bc3108b37c4f service nova] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:28.154823 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df55cd8e-1df5-48cf-8a3e-bc3108b37c4f service nova] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:28.155495 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df55cd8e-1df5-48cf-8a3e-bc3108b37c4f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:28.155838 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-bb5afbcb-ab92-4dd1-a41c-468ff448cf61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:09:28.157752 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-df55cd8e-1df5-48cf-8a3e-bc3108b37c4f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:09:28.174763 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df55cd8e-1df5-48cf-8a3e-bc3108b37c4f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:28.176651 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df55cd8e-1df5-48cf-8a3e-bc3108b37c4f service nova] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:28.189101 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 916/1811] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:09:27 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 281 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 15:09:28.191124 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda10a1e-974a-4767-aacb-c18be0c9fab1 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:28.191846 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda10a1e-974a-4767-aacb-c18be0c9fab1 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:28.192299 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda10a1e-974a-4767-aacb-c18be0c9fab1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:28.216211 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-fda10a1e-974a-4767-aacb-c18be0c9fab1 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:09:28.217685 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-fda10a1e-974a-4767-aacb-c18be0c9fab1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:09:28.248010 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb5afbcb-ab92-4dd1-a41c-468ff448cf61 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:28.260855 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e39bac2-416f-475a-b5c8-b6ea5b4e41ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:28.262340 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e39bac2-416f-475a-b5c8-b6ea5b4e41ed admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:28.264530 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e39bac2-416f-475a-b5c8-b6ea5b4e41ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:28.264866 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e39bac2-416f-475a-b5c8-b6ea5b4e41ed admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:28.265884 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e39bac2-416f-475a-b5c8-b6ea5b4e41ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:28.268788 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e39bac2-416f-475a-b5c8-b6ea5b4e41ed admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:28.296235 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 896/1812] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 15:09:28 2022] GET /identity/v3/roles => generated 1420 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 15:09:28.302657 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e39bac2-416f-475a-b5c8-b6ea5b4e41ed admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:28.326102 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-8e39bac2-416f-475a-b5c8-b6ea5b4e41ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:09:28.335363 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 917/1813] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:09:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:09:28.337103 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda10a1e-974a-4767-aacb-c18be0c9fab1 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:28.347656 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:09:28.381041 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda10a1e-974a-4767-aacb-c18be0c9fab1 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:28.411839 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e39bac2-416f-475a-b5c8-b6ea5b4e41ed admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:28.419832 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89063b45-e277-4c75-ac55-4c39371dae73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:28.423471 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89063b45-e277-4c75-ac55-4c39371dae73 admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:28.423957 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89063b45-e277-4c75-ac55-4c39371dae73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:28.428474 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941aa891-6e1a-462a-a4fc-673639d40467 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:28.430254 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941aa891-6e1a-462a-a4fc-673639d40467 admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:28.430914 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941aa891-6e1a-462a-a4fc-673639d40467 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:28.431191 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941aa891-6e1a-462a-a4fc-673639d40467 admin admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:28.432400 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941aa891-6e1a-462a-a4fc-673639d40467 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:28.438086 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89063b45-e277-4c75-ac55-4c39371dae73 admin admin] PATH_INFO: `/v3/projects/6fb676ab0c6041769602aeaa37855a3c/users/933428759bd04f09b2023413757ec073/roles/a2e551773cbd4df084020676865b8222` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:28.440641 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941aa891-6e1a-462a-a4fc-673639d40467 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:28.470678 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89063b45-e277-4c75-ac55-4c39371dae73 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6fb676ab0c6041769602aeaa37855a3c, user_id=933428759bd04f09b2023413757ec073, role_id=a2e551773cbd4df084020676865b8222)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:28.476262 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89063b45-e277-4c75-ac55-4c39371dae73 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:28.507186 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 918/1814] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 15:09:28 2022] PUT /identity/v3/projects/6fb676ab0c6041769602aeaa37855a3c/users/933428759bd04f09b2023413757ec073/roles/a2e551773cbd4df084020676865b8222 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:09:28.507186 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27789580-ca44-4d89-ab78-49c29fe3954b None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:28.507186 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27789580-ca44-4d89-ab78-49c29fe3954b None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:28.507186 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27789580-ca44-4d89-ab78-49c29fe3954b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:28.521959 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 897/1815] 10.222.0.46 () {60 vars in 1152 bytes} [Thu May 26 15:09:28 2022] POST /identity/v3/projects => generated 367 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 15:09:28.525184 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941aa891-6e1a-462a-a4fc-673639d40467 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:28.532370 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-27789580-ca44-4d89-ab78-49c29fe3954b None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:09:28.533904 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-27789580-ca44-4d89-ab78-49c29fe3954b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:09:28.554596 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:09:28.742748 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941aa891-6e1a-462a-a4fc-673639d40467 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:28.761577 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d426436a-da53-4541-a5f3-8bae10b7292c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:28.762198 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d426436a-da53-4541-a5f3-8bae10b7292c admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:28.762479 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d426436a-da53-4541-a5f3-8bae10b7292c admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:28.763332 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d426436a-da53-4541-a5f3-8bae10b7292c admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:28.764070 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d426436a-da53-4541-a5f3-8bae10b7292c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:28.765766 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d426436a-da53-4541-a5f3-8bae10b7292c admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:28.786939 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 919/1816] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:09:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:09:28.787974 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27789580-ca44-4d89-ab78-49c29fe3954b None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:28.790205 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 898/1817] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 15:09:28 2022] GET /identity/v3/roles => generated 1420 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 15:09:28.791405 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d426436a-da53-4541-a5f3-8bae10b7292c admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:28.819550 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d426436a-da53-4541-a5f3-8bae10b7292c admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:28.819750 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27789580-ca44-4d89-ab78-49c29fe3954b None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:28.834670 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbbde4b-ca6f-4da9-a73b-e4020a3c5345 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:28.835380 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbbde4b-ca6f-4da9-a73b-e4020a3c5345 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:28.835662 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbbde4b-ca6f-4da9-a73b-e4020a3c5345 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:28.835951 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbbde4b-ca6f-4da9-a73b-e4020a3c5345 admin admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:28.836678 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbbde4b-ca6f-4da9-a73b-e4020a3c5345 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 15:09:28.836988 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbbde4b-ca6f-4da9-a73b-e4020a3c5345 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:28.839721 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffbbde4b-ca6f-4da9-a73b-e4020a3c5345 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:28.843817 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887aa6c7-4734-497c-a1f3-acb3349dc7c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:28.844529 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887aa6c7-4734-497c-a1f3-acb3349dc7c1 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:28.844818 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887aa6c7-4734-497c-a1f3-acb3349dc7c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:28.845171 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887aa6c7-4734-497c-a1f3-acb3349dc7c1 admin admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:28.845848 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887aa6c7-4734-497c-a1f3-acb3349dc7c1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:28.853675 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887aa6c7-4734-497c-a1f3-acb3349dc7c1 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:28.883623 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 899/1818] 10.222.0.46 () {58 vars in 1152 bytes} [Thu May 26 15:09:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 15:09:28.885698 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbbde4b-ca6f-4da9-a73b-e4020a3c5345 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:28.903281 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 920/1819] 10.222.0.46 () {60 vars in 1143 bytes} [Thu May 26 15:09:28 2022] POST /identity/v3/users => generated 290 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 15:09:28.906587 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887aa6c7-4734-497c-a1f3-acb3349dc7c1 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:28.915011 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffbbde4b-ca6f-4da9-a73b-e4020a3c5345 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:28.926014 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7cd8aa-1751-41b9-86c5-4d3e75bc42c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:28.927131 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7cd8aa-1751-41b9-86c5-4d3e75bc42c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:28.927412 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7cd8aa-1751-41b9-86c5-4d3e75bc42c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:28.932389 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887aa6c7-4734-497c-a1f3-acb3349dc7c1 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:28.932623 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7cd8aa-1751-41b9-86c5-4d3e75bc42c8 admin admin] PATH_INFO: `/v3/projects/6fb676ab0c6041769602aeaa37855a3c/users/933428759bd04f09b2023413757ec073/roles/a2e551773cbd4df084020676865b8222` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:28.939570 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7cd8aa-1751-41b9-86c5-4d3e75bc42c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6fb676ab0c6041769602aeaa37855a3c, user_id=933428759bd04f09b2023413757ec073, role_id=a2e551773cbd4df084020676865b8222)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:28.953152 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7cd8aa-1751-41b9-86c5-4d3e75bc42c8 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:28.964361 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073f1d8d-03ec-4197-906b-4f4f277b61a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:28.965055 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073f1d8d-03ec-4197-906b-4f4f277b61a9 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:28.965366 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073f1d8d-03ec-4197-906b-4f4f277b61a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:28.966892 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073f1d8d-03ec-4197-906b-4f4f277b61a9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:28.968702 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073f1d8d-03ec-4197-906b-4f4f277b61a9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:28.973654 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073f1d8d-03ec-4197-906b-4f4f277b61a9 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:29.017909 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 900/1820] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 15:09:28 2022] PUT /identity/v3/projects/6fb676ab0c6041769602aeaa37855a3c/users/933428759bd04f09b2023413757ec073/roles/a2e551773cbd4df084020676865b8222 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:09:29.022715 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7cd8aa-1751-41b9-86c5-4d3e75bc42c8 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:29.036295 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 921/1821] 10.222.0.46 () {60 vars in 1152 bytes} [Thu May 26 15:09:28 2022] POST /identity/v3/projects => generated 365 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 15:09:29.037722 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073f1d8d-03ec-4197-906b-4f4f277b61a9 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:29.183561 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073f1d8d-03ec-4197-906b-4f4f277b61a9 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:29.196414 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cb1066-e3a6-405d-82e2-7def8ed960a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:29.197119 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cb1066-e3a6-405d-82e2-7def8ed960a7 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:29.197615 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cb1066-e3a6-405d-82e2-7def8ed960a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:29.198099 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cb1066-e3a6-405d-82e2-7def8ed960a7 admin admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:29.199020 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9cb1066-e3a6-405d-82e2-7def8ed960a7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:29.201223 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9cb1066-e3a6-405d-82e2-7def8ed960a7 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:29.244201 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 922/1822] 10.222.0.46 () {60 vars in 1143 bytes} [Thu May 26 15:09:29 2022] POST /identity/v3/users => generated 283 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 15:09:29.245603 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca471d9d-be73-4de1-8407-886c74d5b5b9 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:29.245897 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca471d9d-be73-4de1-8407-886c74d5b5b9 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:29.246178 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca471d9d-be73-4de1-8407-886c74d5b5b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:29.304994 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7cd8aa-1751-41b9-86c5-4d3e75bc42c8 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:29.311405 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e0cc9a-11fe-470a-95c8-8ed6c59c3296 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:29.312104 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e0cc9a-11fe-470a-95c8-8ed6c59c3296 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:29.312392 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e0cc9a-11fe-470a-95c8-8ed6c59c3296 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:29.312672 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e0cc9a-11fe-470a-95c8-8ed6c59c3296 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:29.313411 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e0cc9a-11fe-470a-95c8-8ed6c59c3296 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:29.315484 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e0cc9a-11fe-470a-95c8-8ed6c59c3296 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:29.317496 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-ca471d9d-be73-4de1-8407-886c74d5b5b9 None None] MFA Rules not processed for user `933428759bd04f09b2023413757ec073`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:09:29.320093 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-ca471d9d-be73-4de1-8407-886c74d5b5b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:09:29.343849 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 901/1823] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 15:09:29 2022] GET /identity/v3/roles => generated 1420 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 15:09:29.350835 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e0cc9a-11fe-470a-95c8-8ed6c59c3296 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:29.363104 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:09:29.364885 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e0cc9a-11fe-470a-95c8-8ed6c59c3296 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:29.378387 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b3b1b9-19a1-4c6d-9b79-bc5aa2158582 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:29.378387 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b3b1b9-19a1-4c6d-9b79-bc5aa2158582 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:29.378387 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b3b1b9-19a1-4c6d-9b79-bc5aa2158582 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:29.378387 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b3b1b9-19a1-4c6d-9b79-bc5aa2158582 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:29.379159 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b3b1b9-19a1-4c6d-9b79-bc5aa2158582 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:29.387089 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b3b1b9-19a1-4c6d-9b79-bc5aa2158582 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:29.418456 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 902/1824] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 15:09:29 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 15:09:29.421296 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b3b1b9-19a1-4c6d-9b79-bc5aa2158582 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:29.451076 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b3b1b9-19a1-4c6d-9b79-bc5aa2158582 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:29.459499 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c9c321-bef8-4b7c-9b20-f0139f7a3a53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:29.461274 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c9c321-bef8-4b7c-9b20-f0139f7a3a53 admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:29.462034 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c9c321-bef8-4b7c-9b20-f0139f7a3a53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:29.462599 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c9c321-bef8-4b7c-9b20-f0139f7a3a53 admin admin] PATH_INFO: `/v3/projects/8834518f444848799f7a09e24fe43f18/users/efb0e1c8fd8a4b1e81826a029fbb0da3/roles/a2e551773cbd4df084020676865b8222` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:29.470524 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 923/1825] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:09:29 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 223 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:09:29.471438 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca471d9d-be73-4de1-8407-886c74d5b5b9 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:29.489051 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca471d9d-be73-4de1-8407-886c74d5b5b9 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:29.489799 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c9c321-bef8-4b7c-9b20-f0139f7a3a53 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8834518f444848799f7a09e24fe43f18, user_id=efb0e1c8fd8a4b1e81826a029fbb0da3, role_id=a2e551773cbd4df084020676865b8222)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:29.492517 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c9c321-bef8-4b7c-9b20-f0139f7a3a53 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:29.498326 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d0a2a3-c406-4668-9769-727f7fa4faa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:29.498326 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d0a2a3-c406-4668-9769-727f7fa4faa6 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:29.498326 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d0a2a3-c406-4668-9769-727f7fa4faa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:29.499150 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d0a2a3-c406-4668-9769-727f7fa4faa6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:29.499837 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d0a2a3-c406-4668-9769-727f7fa4faa6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:29.502593 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d0a2a3-c406-4668-9769-727f7fa4faa6 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:29.517812 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 903/1826] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 15:09:29 2022] PUT /identity/v3/projects/8834518f444848799f7a09e24fe43f18/users/efb0e1c8fd8a4b1e81826a029fbb0da3/roles/a2e551773cbd4df084020676865b8222 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:09:29.521496 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c9c321-bef8-4b7c-9b20-f0139f7a3a53 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:29.521883 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 924/1827] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 15:09:29 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 15:09:29.524397 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-25c9c321-bef8-4b7c-9b20-f0139f7a3a53 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:09:29.527116 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d0a2a3-c406-4668-9769-727f7fa4faa6 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:29.541067 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d0a2a3-c406-4668-9769-727f7fa4faa6 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:29.576714 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:09:29.601360 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c9c321-bef8-4b7c-9b20-f0139f7a3a53 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:29.607482 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847d7781-03c8-4b02-a0dd-4c2c60548683 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5b5b35614a547d783e6e77ddb8c1c03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:29.608593 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847d7781-03c8-4b02-a0dd-4c2c60548683 service cinder] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:29.609143 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847d7781-03c8-4b02-a0dd-4c2c60548683 service cinder] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:29.609577 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847d7781-03c8-4b02-a0dd-4c2c60548683 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:29.803542 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847d7781-03c8-4b02-a0dd-4c2c60548683 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:29.806179 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847d7781-03c8-4b02-a0dd-4c2c60548683 service cinder] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:29.839622 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 904/1828] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:09:29 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 15:09:29.840703 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847d7781-03c8-4b02-a0dd-4c2c60548683 service cinder] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:29.874360 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f96f04-c697-45bd-8846-f95b42ce56d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:29.875422 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f96f04-c697-45bd-8846-f95b42ce56d9 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:29.876120 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f96f04-c697-45bd-8846-f95b42ce56d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:29.877740 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f96f04-c697-45bd-8846-f95b42ce56d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:29.878863 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f96f04-c697-45bd-8846-f95b42ce56d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:29.881501 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f96f04-c697-45bd-8846-f95b42ce56d9 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:29.911176 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 925/1829] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 15:09:29 2022] GET /identity/v3/roles => generated 1420 bytes in 385 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 15:09:29.915239 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f96f04-c697-45bd-8846-f95b42ce56d9 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:29.935907 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f96f04-c697-45bd-8846-f95b42ce56d9 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:29.948472 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9bcc04-4f42-4815-9da1-a030d9244ce5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:29.952502 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9bcc04-4f42-4815-9da1-a030d9244ce5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:29.954456 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9bcc04-4f42-4815-9da1-a030d9244ce5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:29.955179 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9bcc04-4f42-4815-9da1-a030d9244ce5 admin admin] PATH_INFO: `/v3/projects/8834518f444848799f7a09e24fe43f18/users/efb0e1c8fd8a4b1e81826a029fbb0da3/roles/a2e551773cbd4df084020676865b8222` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:29.971051 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9bcc04-4f42-4815-9da1-a030d9244ce5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8834518f444848799f7a09e24fe43f18, user_id=efb0e1c8fd8a4b1e81826a029fbb0da3, role_id=a2e551773cbd4df084020676865b8222)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:29.976165 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847d7781-03c8-4b02-a0dd-4c2c60548683 service cinder] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:29.978717 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9bcc04-4f42-4815-9da1-a030d9244ce5 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:29.988028 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258ae79c-98af-4d02-af32-188dd47e2aac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:29.990515 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258ae79c-98af-4d02-af32-188dd47e2aac admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:29.991459 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258ae79c-98af-4d02-af32-188dd47e2aac admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:29.991923 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258ae79c-98af-4d02-af32-188dd47e2aac admin admin] PATH_INFO: `/v3/projects/d698e430e3c84396a3c925c8732e9b84/users/deeec9f13fe74bf2b8bd37db9c41d9a9/roles/d4088d1845a24fbd9bd8975ce0b1d3d0` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:29.994862 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 926/1830] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 15:09:29 2022] PUT /identity/v3/projects/8834518f444848799f7a09e24fe43f18/users/efb0e1c8fd8a4b1e81826a029fbb0da3/roles/a2e551773cbd4df084020676865b8222 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:09:30.007445 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9bcc04-4f42-4815-9da1-a030d9244ce5 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:30.047683 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258ae79c-98af-4d02-af32-188dd47e2aac admin admin] RBAC: Authorizing `identity:create_grant(project_id=d698e430e3c84396a3c925c8732e9b84, user_id=deeec9f13fe74bf2b8bd37db9c41d9a9, role_id=d4088d1845a24fbd9bd8975ce0b1d3d0)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:30.056144 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258ae79c-98af-4d02-af32-188dd47e2aac admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:30.101839 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 905/1831] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 15:09:29 2022] PUT /identity/v3/projects/d698e430e3c84396a3c925c8732e9b84/users/deeec9f13fe74bf2b8bd37db9c41d9a9/roles/d4088d1845a24fbd9bd8975ce0b1d3d0 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:09:30.103767 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb8e1ed-585d-45a3-a930-1d06629f4a0a None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:30.104517 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb8e1ed-585d-45a3-a930-1d06629f4a0a None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:30.104994 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb8e1ed-585d-45a3-a930-1d06629f4a0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:30.127673 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9bcc04-4f42-4815-9da1-a030d9244ce5 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:30.202401 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-dfb8e1ed-585d-45a3-a930-1d06629f4a0a None None] MFA Rules not processed for user `efb0e1c8fd8a4b1e81826a029fbb0da3`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:09:30.207007 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-dfb8e1ed-585d-45a3-a930-1d06629f4a0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:09:30.212522 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:09:30.238150 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db880f69-37cb-49a2-b26d-583dd1b440e9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5b5b35614a547d783e6e77ddb8c1c03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:30.241236 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db880f69-37cb-49a2-b26d-583dd1b440e9 service cinder] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:30.241810 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db880f69-37cb-49a2-b26d-583dd1b440e9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:30.242282 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db880f69-37cb-49a2-b26d-583dd1b440e9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:30.251584 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-db880f69-37cb-49a2-b26d-583dd1b440e9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:09:30.354856 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:09:30.378382 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 906/1832] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:09:30 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 276 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:09:30.384138 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb77afa-8aaa-465d-998d-957fa7e8e36d None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:30.384438 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb77afa-8aaa-465d-998d-957fa7e8e36d None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:30.384723 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb77afa-8aaa-465d-998d-957fa7e8e36d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:30.449372 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-9fb77afa-8aaa-465d-998d-957fa7e8e36d None None] MFA Rules not processed for user `deeec9f13fe74bf2b8bd37db9c41d9a9`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:09:30.451391 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-9fb77afa-8aaa-465d-998d-957fa7e8e36d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:09:30.468324 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db880f69-37cb-49a2-b26d-583dd1b440e9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:30.471668 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db880f69-37cb-49a2-b26d-583dd1b440e9 service cinder] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:30.490831 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 927/1833] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:09:30 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 484 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 15:09:30.532800 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db880f69-37cb-49a2-b26d-583dd1b440e9 service cinder] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:30.535519 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-db880f69-37cb-49a2-b26d-583dd1b440e9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:09:30.568091 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db880f69-37cb-49a2-b26d-583dd1b440e9 service cinder] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:30.592838 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb0a856-e308-45ab-8e7c-1ac57069cbad service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5b5b35614a547d783e6e77ddb8c1c03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:30.594753 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb0a856-e308-45ab-8e7c-1ac57069cbad service cinder] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:30.594753 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb0a856-e308-45ab-8e7c-1ac57069cbad service cinder] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:30.597493 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb0a856-e308-45ab-8e7c-1ac57069cbad service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:30.629162 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 907/1834] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:09:30 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:09:30.635115 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb77afa-8aaa-465d-998d-957fa7e8e36d None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:30.644567 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb0a856-e308-45ab-8e7c-1ac57069cbad service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:30.647483 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb0a856-e308-45ab-8e7c-1ac57069cbad service cinder] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:30.669912 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 928/1835] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:09:30 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 15:09:30.777144 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb77afa-8aaa-465d-998d-957fa7e8e36d None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:30.790356 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f62c41d-568b-4bfb-bf86-324665730f09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:30.791043 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f62c41d-568b-4bfb-bf86-324665730f09 admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:30.791564 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f62c41d-568b-4bfb-bf86-324665730f09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:30.791739 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f62c41d-568b-4bfb-bf86-324665730f09 admin admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:30.792300 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f62c41d-568b-4bfb-bf86-324665730f09 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:30.794821 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f62c41d-568b-4bfb-bf86-324665730f09 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:30.866017 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 908/1836] 10.222.0.46 () {60 vars in 1152 bytes} [Thu May 26 15:09:30 2022] POST /identity/v3/projects => generated 365 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 15:09:30.872723 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb0a856-e308-45ab-8e7c-1ac57069cbad service cinder] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:30.900055 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb0a856-e308-45ab-8e7c-1ac57069cbad service cinder] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:31.100986 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f62c41d-568b-4bfb-bf86-324665730f09 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:31.157584 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a607ccfe-75b3-43e4-af67-5f4ec70138a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:31.158243 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a607ccfe-75b3-43e4-af67-5f4ec70138a9 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:31.158506 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a607ccfe-75b3-43e4-af67-5f4ec70138a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:31.158789 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a607ccfe-75b3-43e4-af67-5f4ec70138a9 admin admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:31.159498 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a607ccfe-75b3-43e4-af67-5f4ec70138a9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:31.165116 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a607ccfe-75b3-43e4-af67-5f4ec70138a9 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:31.216788 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 929/1837] 10.222.0.46 () {60 vars in 1143 bytes} [Thu May 26 15:09:30 2022] POST /identity/v3/users => generated 289 bytes in 345 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 15:09:31.220542 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a607ccfe-75b3-43e4-af67-5f4ec70138a9 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:31.238463 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f62c41d-568b-4bfb-bf86-324665730f09 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:31.238871 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a607ccfe-75b3-43e4-af67-5f4ec70138a9 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:31.245651 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd4409f-e620-4ba2-b27b-67e53421fffd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:31.246498 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd4409f-e620-4ba2-b27b-67e53421fffd admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:31.246772 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd4409f-e620-4ba2-b27b-67e53421fffd admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:31.247630 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd4409f-e620-4ba2-b27b-67e53421fffd admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:31.248109 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd62bb1-233b-43f2-a7fb-180cda018df8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5b5b35614a547d783e6e77ddb8c1c03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:31.248370 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd4409f-e620-4ba2-b27b-67e53421fffd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:31.249240 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd62bb1-233b-43f2-a7fb-180cda018df8 service cinder] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:31.250894 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd4409f-e620-4ba2-b27b-67e53421fffd admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:31.251356 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd62bb1-233b-43f2-a7fb-180cda018df8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:31.251920 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd62bb1-233b-43f2-a7fb-180cda018df8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:31.284847 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 930/1838] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 15:09:31 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 15:09:31.286042 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd4409f-e620-4ba2-b27b-67e53421fffd admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:31.297877 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd4409f-e620-4ba2-b27b-67e53421fffd admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:31.303723 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a12f6b7-3a73-462d-81c6-ad7d7e88d34d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:31.304824 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a12f6b7-3a73-462d-81c6-ad7d7e88d34d admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:31.305315 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a12f6b7-3a73-462d-81c6-ad7d7e88d34d admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:31.305924 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a12f6b7-3a73-462d-81c6-ad7d7e88d34d admin admin] PATH_INFO: `/v3/projects/67808d9e84714011b25197dbeeeb989d/users/de10f500a75942d0b7486006b0f2e49f/roles/a2e551773cbd4df084020676865b8222` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:31.322017 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:09:31.324665 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a12f6b7-3a73-462d-81c6-ad7d7e88d34d admin admin] RBAC: Authorizing `identity:create_grant(project_id=67808d9e84714011b25197dbeeeb989d, user_id=de10f500a75942d0b7486006b0f2e49f, role_id=a2e551773cbd4df084020676865b8222)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:31.327752 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a12f6b7-3a73-462d-81c6-ad7d7e88d34d admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:31.350012 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 931/1839] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 15:09:31 2022] PUT /identity/v3/projects/67808d9e84714011b25197dbeeeb989d/users/de10f500a75942d0b7486006b0f2e49f/roles/a2e551773cbd4df084020676865b8222 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:09:31.353796 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a12f6b7-3a73-462d-81c6-ad7d7e88d34d admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:31.414428 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd62bb1-233b-43f2-a7fb-180cda018df8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:31.416412 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd62bb1-233b-43f2-a7fb-180cda018df8 service cinder] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:31.428144 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:09:31.455646 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 909/1840] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:09:31 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 15:09:31.459515 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a12f6b7-3a73-462d-81c6-ad7d7e88d34d admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:31.475102 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d88ec8-92a8-40ff-bb92-cf556a5b0c1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:31.477382 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d88ec8-92a8-40ff-bb92-cf556a5b0c1e admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:31.477701 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d88ec8-92a8-40ff-bb92-cf556a5b0c1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:31.478014 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d88ec8-92a8-40ff-bb92-cf556a5b0c1e admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:31.478771 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d88ec8-92a8-40ff-bb92-cf556a5b0c1e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:31.480423 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d88ec8-92a8-40ff-bb92-cf556a5b0c1e admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:31.510425 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 932/1841] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 15:09:31 2022] GET /identity/v3/roles => generated 1420 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 15:09:31.518772 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd62bb1-233b-43f2-a7fb-180cda018df8 service cinder] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:31.534907 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd62bb1-233b-43f2-a7fb-180cda018df8 service cinder] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:31.542154 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23eaa01e-70be-469b-86af-2d2df6ca7bb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:31.543277 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23eaa01e-70be-469b-86af-2d2df6ca7bb8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:31.544454 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23eaa01e-70be-469b-86af-2d2df6ca7bb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:31.544999 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23eaa01e-70be-469b-86af-2d2df6ca7bb8 admin admin] PATH_INFO: `/v3/projects/67808d9e84714011b25197dbeeeb989d/users/de10f500a75942d0b7486006b0f2e49f/roles/a2e551773cbd4df084020676865b8222` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:31.559121 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23eaa01e-70be-469b-86af-2d2df6ca7bb8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=67808d9e84714011b25197dbeeeb989d, user_id=de10f500a75942d0b7486006b0f2e49f, role_id=a2e551773cbd4df084020676865b8222)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:31.567040 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23eaa01e-70be-469b-86af-2d2df6ca7bb8 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:31.609932 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 910/1842] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 15:09:31 2022] PUT /identity/v3/projects/67808d9e84714011b25197dbeeeb989d/users/de10f500a75942d0b7486006b0f2e49f/roles/a2e551773cbd4df084020676865b8222 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:09:31.610897 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6a42bb-2a28-4a8f-9d1b-d6c2518038d6 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:31.611201 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6a42bb-2a28-4a8f-9d1b-d6c2518038d6 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:31.611997 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6a42bb-2a28-4a8f-9d1b-d6c2518038d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:31.682743 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-be6a42bb-2a28-4a8f-9d1b-d6c2518038d6 None None] MFA Rules not processed for user `de10f500a75942d0b7486006b0f2e49f`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:09:31.689125 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-be6a42bb-2a28-4a8f-9d1b-d6c2518038d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:09:31.948531 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 933/1843] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:09:31 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 339 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:09:32.058321 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23eaa01e-70be-469b-86af-2d2df6ca7bb8 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:32.241723 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23eaa01e-70be-469b-86af-2d2df6ca7bb8 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:32.255173 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68840f5b-5731-4d83-90e7-9b0bdd0e90cc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5b5b35614a547d783e6e77ddb8c1c03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:32.256652 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68840f5b-5731-4d83-90e7-9b0bdd0e90cc service cinder] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:32.256947 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68840f5b-5731-4d83-90e7-9b0bdd0e90cc service cinder] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:32.257227 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68840f5b-5731-4d83-90e7-9b0bdd0e90cc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:32.286559 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68840f5b-5731-4d83-90e7-9b0bdd0e90cc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:32.290941 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68840f5b-5731-4d83-90e7-9b0bdd0e90cc service cinder] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:32.341223 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 911/1844] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:09:32 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 283 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 15:09:32.454206 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6a42bb-2a28-4a8f-9d1b-d6c2518038d6 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:32.468268 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6a42bb-2a28-4a8f-9d1b-d6c2518038d6 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:32.474836 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d2f5e2-d5ac-4186-a840-3da9be72940b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5b5b35614a547d783e6e77ddb8c1c03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:32.476273 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d2f5e2-d5ac-4186-a840-3da9be72940b service cinder] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:32.478302 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d2f5e2-d5ac-4186-a840-3da9be72940b service cinder] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:32.478824 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d2f5e2-d5ac-4186-a840-3da9be72940b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:32.577288 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d2f5e2-d5ac-4186-a840-3da9be72940b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:32.586543 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d2f5e2-d5ac-4186-a840-3da9be72940b service cinder] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:32.617916 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 934/1845] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:09:32 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 15:09:33.374136 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68840f5b-5731-4d83-90e7-9b0bdd0e90cc service cinder] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:33.375914 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-68840f5b-5731-4d83-90e7-9b0bdd0e90cc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:09:33.598553 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68840f5b-5731-4d83-90e7-9b0bdd0e90cc service cinder] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:33.608076 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49605153-9bea-43b1-a6ee-9534bd5a466f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:33.609604 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49605153-9bea-43b1-a6ee-9534bd5a466f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:33.610393 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49605153-9bea-43b1-a6ee-9534bd5a466f admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:33.610996 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49605153-9bea-43b1-a6ee-9534bd5a466f admin admin] PATH_INFO: `/v3/users/c431cf6affab4ba7a916ff8b829cd7b7` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:33.612666 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49605153-9bea-43b1-a6ee-9534bd5a466f admin admin] RBAC: Authorizing `identity:delete_user(user_id=c431cf6affab4ba7a916ff8b829cd7b7)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:33.628523 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49605153-9bea-43b1-a6ee-9534bd5a466f admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:33.712511 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-49605153-9bea-43b1-a6ee-9534bd5a466f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c431cf6affab4ba7a916ff8b829cd7b7'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:33.737306 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-49605153-9bea-43b1-a6ee-9534bd5a466f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c431cf6affab4ba7a916ff8b829cd7b7'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:33.740595 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-49605153-9bea-43b1-a6ee-9534bd5a466f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c431cf6affab4ba7a916ff8b829cd7b7'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:33.748983 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:09:33.785380 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-49605153-9bea-43b1-a6ee-9534bd5a466f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c431cf6affab4ba7a916ff8b829cd7b7'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:33.822921 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-49605153-9bea-43b1-a6ee-9534bd5a466f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c431cf6affab4ba7a916ff8b829cd7b7'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:33.848095 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 912/1846] 10.222.0.46 () {58 vars in 1223 bytes} [Thu May 26 15:09:33 2022] DELETE /identity/v3/users/c431cf6affab4ba7a916ff8b829cd7b7 => generated 0 bytes in 475 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:09:33.853185 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa531c04-b4a7-4636-8724-14d591b01d7a None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:33.853701 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa531c04-b4a7-4636-8724-14d591b01d7a None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:33.853886 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa531c04-b4a7-4636-8724-14d591b01d7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:33.891085 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-fa531c04-b4a7-4636-8724-14d591b01d7a None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:09:33.895172 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-fa531c04-b4a7-4636-8724-14d591b01d7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:09:34.076905 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:09:34.113506 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 935/1847] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:09:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:09:34.208403 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49605153-9bea-43b1-a6ee-9534bd5a466f admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:34.209677 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-49605153-9bea-43b1-a6ee-9534bd5a466f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:09:34.309157 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3cd8be-906b-4780-a0ed-0f05c7a9a2db None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:34.309477 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3cd8be-906b-4780-a0ed-0f05c7a9a2db None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:34.309938 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3cd8be-906b-4780-a0ed-0f05c7a9a2db None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:34.311949 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 936/1848] 10.222.0.46 () {54 vars in 921 bytes} [Thu May 26 15:09:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 15:09:34.345311 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49605153-9bea-43b1-a6ee-9534bd5a466f admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:34.346316 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80ef0e6-48d7-48c1-9f16-888df011d6c4 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:34.346616 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80ef0e6-48d7-48c1-9f16-888df011d6c4 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:34.346925 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a80ef0e6-48d7-48c1-9f16-888df011d6c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:34.362034 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7281b87-8847-4627-86af-b3a4d2333c79 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0156a2f6f3794f479e76b85bdbbe5c26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:34.363044 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7281b87-8847-4627-86af-b3a4d2333c79 service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:34.363515 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7281b87-8847-4627-86af-b3a4d2333c79 service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:34.364045 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7281b87-8847-4627-86af-b3a4d2333c79 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:34.372547 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-a80ef0e6-48d7-48c1-9f16-888df011d6c4 None None] MFA Rules not processed for user `d6639353c70449d1b3d10cd6487c3b00`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:09:34.374185 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-a80ef0e6-48d7-48c1-9f16-888df011d6c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:09:34.389764 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7281b87-8847-4627-86af-b3a4d2333c79 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:34.391798 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7281b87-8847-4627-86af-b3a4d2333c79 service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:34.432609 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 913/1849] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:09:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 15:09:34.561495 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 937/1850] 10.222.0.46 () {58 vars in 1018 bytes} [Thu May 26 15:09:34 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:09:34.631439 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7281b87-8847-4627-86af-b3a4d2333c79 service neutron] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:34.632694 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-b7281b87-8847-4627-86af-b3a4d2333c79 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:09:34.664924 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7281b87-8847-4627-86af-b3a4d2333c79 service neutron] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:34.673996 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1b3ef0-3f11-488f-a84b-17b6fb258de5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6639353c70449d1b3d10cd6487c3b00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:34.674695 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1b3ef0-3f11-488f-a84b-17b6fb258de5 service nova] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:34.675012 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1b3ef0-3f11-488f-a84b-17b6fb258de5 service nova] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:34.675451 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1b3ef0-3f11-488f-a84b-17b6fb258de5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:34.708639 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1b3ef0-3f11-488f-a84b-17b6fb258de5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:34.710310 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1b3ef0-3f11-488f-a84b-17b6fb258de5 service nova] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:34.724970 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 914/1851] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:09:34 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 15:09:35.718904 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80ef0e6-48d7-48c1-9f16-888df011d6c4 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:35.720224 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-a80ef0e6-48d7-48c1-9f16-888df011d6c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:09:35.739826 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a80ef0e6-48d7-48c1-9f16-888df011d6c4 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:35.751282 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1b3ef0-3f11-488f-a84b-17b6fb258de5 service nova] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:35.753029 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-7d1b3ef0-3f11-488f-a84b-17b6fb258de5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:09:35.760009 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7175695e-8fb4-49dc-9156-fead6eaa82a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:35.761682 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7175695e-8fb4-49dc-9156-fead6eaa82a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:35.763173 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7175695e-8fb4-49dc-9156-fead6eaa82a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:35.763620 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7175695e-8fb4-49dc-9156-fead6eaa82a6 admin admin] PATH_INFO: `/v3/projects/edc7098ee470478e8c24a8affc9d5161` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:35.768118 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7175695e-8fb4-49dc-9156-fead6eaa82a6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=edc7098ee470478e8c24a8affc9d5161)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:35.773816 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7175695e-8fb4-49dc-9156-fead6eaa82a6 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:35.785851 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1b3ef0-3f11-488f-a84b-17b6fb258de5 service nova] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:35.797200 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d2a9c2-6089-4491-b7f7-183974d86315 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:35.798131 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d2a9c2-6089-4491-b7f7-183974d86315 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:35.798416 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d2a9c2-6089-4491-b7f7-183974d86315 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:35.798693 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d2a9c2-6089-4491-b7f7-183974d86315 admin admin] PATH_INFO: `/v3/users/fe57a7ef56fb4437b4c981b312ab8c7d` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:35.805961 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d2a9c2-6089-4491-b7f7-183974d86315 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fe57a7ef56fb4437b4c981b312ab8c7d)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:35.808053 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d2a9c2-6089-4491-b7f7-183974d86315 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:35.944415 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-7175695e-8fb4-49dc-9156-fead6eaa82a6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'edc7098ee470478e8c24a8affc9d5161', 'request_id': 'req-7175695e-8fb4-49dc-9156-fead6eaa82a6'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:35.947380 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-53d2a9c2-6089-4491-b7f7-183974d86315 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fe57a7ef56fb4437b4c981b312ab8c7d'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:35.963161 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-7175695e-8fb4-49dc-9156-fead6eaa82a6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'edc7098ee470478e8c24a8affc9d5161', 'request_id': 'req-7175695e-8fb4-49dc-9156-fead6eaa82a6'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:35.978406 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-53d2a9c2-6089-4491-b7f7-183974d86315 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fe57a7ef56fb4437b4c981b312ab8c7d'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:35.979588 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-53d2a9c2-6089-4491-b7f7-183974d86315 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fe57a7ef56fb4437b4c981b312ab8c7d'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:35.983515 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-7175695e-8fb4-49dc-9156-fead6eaa82a6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'edc7098ee470478e8c24a8affc9d5161', 'request_id': 'req-7175695e-8fb4-49dc-9156-fead6eaa82a6'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:36.013263 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-53d2a9c2-6089-4491-b7f7-183974d86315 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fe57a7ef56fb4437b4c981b312ab8c7d'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:36.024972 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-7175695e-8fb4-49dc-9156-fead6eaa82a6 admin admin] The token cache is being invalidate because project edc7098ee470478e8c24a8affc9d5161 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 15:09:36.026614 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-7175695e-8fb4-49dc-9156-fead6eaa82a6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:36.027496 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-7175695e-8fb4-49dc-9156-fead6eaa82a6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:36.031406 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 938/1852] 10.222.0.46 () {58 vars in 1232 bytes} [Thu May 26 15:09:35 2022] DELETE /identity/v3/projects/edc7098ee470478e8c24a8affc9d5161 => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:09:36.037169 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9308b96-5c45-475a-8666-d1896c3758f3 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:36.037487 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9308b96-5c45-475a-8666-d1896c3758f3 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:36.039760 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9308b96-5c45-475a-8666-d1896c3758f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:36.047971 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-53d2a9c2-6089-4491-b7f7-183974d86315 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fe57a7ef56fb4437b4c981b312ab8c7d'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:36.066888 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 915/1853] 10.222.0.46 () {58 vars in 1223 bytes} [Thu May 26 15:09:35 2022] DELETE /identity/v3/users/fe57a7ef56fb4437b4c981b312ab8c7d => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 15:09:36.068473 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3038032b-213d-4dd7-8497-5fb10a0ee473 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:36.069019 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3038032b-213d-4dd7-8497-5fb10a0ee473 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:36.069641 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3038032b-213d-4dd7-8497-5fb10a0ee473 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:36.073309 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-e9308b96-5c45-475a-8666-d1896c3758f3 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:09:36.075634 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-e9308b96-5c45-475a-8666-d1896c3758f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:09:36.119233 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-3038032b-213d-4dd7-8497-5fb10a0ee473 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:09:36.120620 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-3038032b-213d-4dd7-8497-5fb10a0ee473 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:09:36.174273 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:09:36.187468 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 939/1854] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:09:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 15:09:36.189667 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9308b96-5c45-475a-8666-d1896c3758f3 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:36.191386 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-e9308b96-5c45-475a-8666-d1896c3758f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:09:36.239908 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9308b96-5c45-475a-8666-d1896c3758f3 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:36.246229 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70fe1b5-5977-48ce-9794-cafcd05eca39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:36.247697 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70fe1b5-5977-48ce-9794-cafcd05eca39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:36.247986 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70fe1b5-5977-48ce-9794-cafcd05eca39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:36.248261 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70fe1b5-5977-48ce-9794-cafcd05eca39 admin admin] PATH_INFO: `/v3/users/deeec9f13fe74bf2b8bd37db9c41d9a9` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:36.249890 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70fe1b5-5977-48ce-9794-cafcd05eca39 admin admin] RBAC: Authorizing `identity:delete_user(user_id=deeec9f13fe74bf2b8bd37db9c41d9a9)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:36.251999 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70fe1b5-5977-48ce-9794-cafcd05eca39 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:36.262050 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 916/1855] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:09:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:09:36.264451 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c48538-fa4d-471d-9536-b5b61ab105e6 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:36.267627 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c48538-fa4d-471d-9536-b5b61ab105e6 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:36.267926 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c48538-fa4d-471d-9536-b5b61ab105e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:36.275115 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:09:36.297286 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-73c48538-fa4d-471d-9536-b5b61ab105e6 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:09:36.300691 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-73c48538-fa4d-471d-9536-b5b61ab105e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:09:36.315080 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 917/1856] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:09:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:09:36.316902 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c48538-fa4d-471d-9536-b5b61ab105e6 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:36.321184 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-73c48538-fa4d-471d-9536-b5b61ab105e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:09:36.367585 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-f70fe1b5-5977-48ce-9794-cafcd05eca39 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'deeec9f13fe74bf2b8bd37db9c41d9a9'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:36.384520 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-f70fe1b5-5977-48ce-9794-cafcd05eca39 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'deeec9f13fe74bf2b8bd37db9c41d9a9'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:36.385268 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-f70fe1b5-5977-48ce-9794-cafcd05eca39 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'deeec9f13fe74bf2b8bd37db9c41d9a9'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:36.420713 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-f70fe1b5-5977-48ce-9794-cafcd05eca39 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'deeec9f13fe74bf2b8bd37db9c41d9a9'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:36.432542 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-f70fe1b5-5977-48ce-9794-cafcd05eca39 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'deeec9f13fe74bf2b8bd37db9c41d9a9'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:36.443666 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 940/1857] 10.222.0.46 () {58 vars in 1223 bytes} [Thu May 26 15:09:36 2022] DELETE /identity/v3/users/deeec9f13fe74bf2b8bd37db9c41d9a9 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:09:36.447541 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70fe1b5-5977-48ce-9794-cafcd05eca39 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:36.450494 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-f70fe1b5-5977-48ce-9794-cafcd05eca39 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:09:36.456416 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c48538-fa4d-471d-9536-b5b61ab105e6 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:36.583691 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70fe1b5-5977-48ce-9794-cafcd05eca39 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:36.596977 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e433da08-4c76-4535-8d0b-7bf85a25de09 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0156a2f6f3794f479e76b85bdbbe5c26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:36.597697 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e433da08-4c76-4535-8d0b-7bf85a25de09 service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:36.597983 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e433da08-4c76-4535-8d0b-7bf85a25de09 service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:36.600548 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19aea8e7-c5b4-4b60-8816-4d19940f4447 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:36.601249 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19aea8e7-c5b4-4b60-8816-4d19940f4447 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:36.601569 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19aea8e7-c5b4-4b60-8816-4d19940f4447 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:36.601855 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19aea8e7-c5b4-4b60-8816-4d19940f4447 admin admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:36.602604 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19aea8e7-c5b4-4b60-8816-4d19940f4447 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 15:09:36.602915 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19aea8e7-c5b4-4b60-8816-4d19940f4447 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:36.608079 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e433da08-4c76-4535-8d0b-7bf85a25de09 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:36.611573 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-e433da08-4c76-4535-8d0b-7bf85a25de09 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:09:36.614060 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19aea8e7-c5b4-4b60-8816-4d19940f4447 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:36.644420 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 941/1858] 10.222.0.46 () {58 vars in 1152 bytes} [Thu May 26 15:09:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 15:09:36.646402 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42501331-aaeb-40a0-aa2f-7bf971aa844f None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:36.647119 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42501331-aaeb-40a0-aa2f-7bf971aa844f None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:36.647582 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42501331-aaeb-40a0-aa2f-7bf971aa844f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:36.653835 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e433da08-4c76-4535-8d0b-7bf85a25de09 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:36.655326 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e433da08-4c76-4535-8d0b-7bf85a25de09 service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:36.672412 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-42501331-aaeb-40a0-aa2f-7bf971aa844f None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:09:36.674244 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-42501331-aaeb-40a0-aa2f-7bf971aa844f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:09:36.684836 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 942/1859] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:09:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:09:36.685976 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42501331-aaeb-40a0-aa2f-7bf971aa844f None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:36.695932 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 918/1860] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:09:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 15:09:36.704997 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42501331-aaeb-40a0-aa2f-7bf971aa844f None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:36.721499 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6792c15b-e6e9-44b0-bf50-c42e8baf8752 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:36.722363 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6792c15b-e6e9-44b0-bf50-c42e8baf8752 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:36.722619 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6792c15b-e6e9-44b0-bf50-c42e8baf8752 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:36.722907 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6792c15b-e6e9-44b0-bf50-c42e8baf8752 admin admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:36.723651 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6792c15b-e6e9-44b0-bf50-c42e8baf8752 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:36.725718 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6792c15b-e6e9-44b0-bf50-c42e8baf8752 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:36.736509 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e433da08-4c76-4535-8d0b-7bf85a25de09 service neutron] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:36.738513 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-e433da08-4c76-4535-8d0b-7bf85a25de09 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:09:36.755050 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e433da08-4c76-4535-8d0b-7bf85a25de09 service neutron] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:36.768747 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107143e1-ef63-46bd-9b4f-1aacdd66210e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0156a2f6f3794f479e76b85bdbbe5c26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:36.769857 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107143e1-ef63-46bd-9b4f-1aacdd66210e service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:36.770365 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107143e1-ef63-46bd-9b4f-1aacdd66210e service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:36.771999 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107143e1-ef63-46bd-9b4f-1aacdd66210e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:36.790015 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 943/1861] 10.222.0.46 () {60 vars in 1152 bytes} [Thu May 26 15:09:36 2022] POST /identity/v3/projects => generated 377 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 15:09:36.790971 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6792c15b-e6e9-44b0-bf50-c42e8baf8752 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:36.795295 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107143e1-ef63-46bd-9b4f-1aacdd66210e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:36.799991 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107143e1-ef63-46bd-9b4f-1aacdd66210e service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:36.819919 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 919/1862] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:09:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 15:09:36.893267 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:DELETE=10,keystone:SELECT=88,keystone:INSERT=2 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:09:36.912932 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6792c15b-e6e9-44b0-bf50-c42e8baf8752 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:36.921588 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04098d9-8548-4fca-9675-33123d82486d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:36.922722 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04098d9-8548-4fca-9675-33123d82486d admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:36.924555 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04098d9-8548-4fca-9675-33123d82486d admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:36.924555 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04098d9-8548-4fca-9675-33123d82486d admin admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:36.924555 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04098d9-8548-4fca-9675-33123d82486d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:36.926883 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04098d9-8548-4fca-9675-33123d82486d admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:36.962333 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 944/1863] 10.222.0.46 () {60 vars in 1143 bytes} [Thu May 26 15:09:36 2022] POST /identity/v3/users => generated 289 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 15:09:36.966414 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107143e1-ef63-46bd-9b4f-1aacdd66210e service neutron] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:36.978037 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107143e1-ef63-46bd-9b4f-1aacdd66210e service neutron] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:37.000128 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cdb0ef-9113-4214-a782-6a2236e7cb17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:37.005102 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cdb0ef-9113-4214-a782-6a2236e7cb17 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:37.005629 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cdb0ef-9113-4214-a782-6a2236e7cb17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:37.009039 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cdb0ef-9113-4214-a782-6a2236e7cb17 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:37.010270 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69cdb0ef-9113-4214-a782-6a2236e7cb17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:37.013052 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69cdb0ef-9113-4214-a782-6a2236e7cb17 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:37.041829 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 920/1864] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 15:09:36 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 15:09:37.046286 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04098d9-8548-4fca-9675-33123d82486d admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:37.066560 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04098d9-8548-4fca-9675-33123d82486d admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:37.093568 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa31cbaa-fd86-433e-baaa-bb8c4c6ffbab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:37.097810 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa31cbaa-fd86-433e-baaa-bb8c4c6ffbab admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:37.098143 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa31cbaa-fd86-433e-baaa-bb8c4c6ffbab admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:37.098644 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa31cbaa-fd86-433e-baaa-bb8c4c6ffbab admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:37.099486 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa31cbaa-fd86-433e-baaa-bb8c4c6ffbab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:37.101500 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa31cbaa-fd86-433e-baaa-bb8c4c6ffbab admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:37.136581 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 945/1865] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 15:09:37 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 15:09:37.141717 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cdb0ef-9113-4214-a782-6a2236e7cb17 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:37.156075 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cdb0ef-9113-4214-a782-6a2236e7cb17 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:37.163064 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5899d783-1c75-4fe1-98b0-75df5c062924 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:37.172144 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5899d783-1c75-4fe1-98b0-75df5c062924 admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:37.172976 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5899d783-1c75-4fe1-98b0-75df5c062924 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:37.173660 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5899d783-1c75-4fe1-98b0-75df5c062924 admin admin] PATH_INFO: `/v3/projects/156f43da4ee5406abc05ecc0d89e9042/users/79615a15f7634b1fbfa2172227b2d39b/roles/d4088d1845a24fbd9bd8975ce0b1d3d0` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:37.200596 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5899d783-1c75-4fe1-98b0-75df5c062924 admin admin] RBAC: Authorizing `identity:create_grant(project_id=156f43da4ee5406abc05ecc0d89e9042, user_id=79615a15f7634b1fbfa2172227b2d39b, role_id=d4088d1845a24fbd9bd8975ce0b1d3d0)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:37.206327 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5899d783-1c75-4fe1-98b0-75df5c062924 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:37.249795 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 921/1866] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 15:09:37 2022] PUT /identity/v3/projects/156f43da4ee5406abc05ecc0d89e9042/users/79615a15f7634b1fbfa2172227b2d39b/roles/d4088d1845a24fbd9bd8975ce0b1d3d0 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:09:37.260715 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776d1537-bd70-4b30-a967-94cdff7b5e0c None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:37.261017 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776d1537-bd70-4b30-a967-94cdff7b5e0c None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:37.261344 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-776d1537-bd70-4b30-a967-94cdff7b5e0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:37.299608 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-776d1537-bd70-4b30-a967-94cdff7b5e0c None None] MFA Rules not processed for user `79615a15f7634b1fbfa2172227b2d39b`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:09:37.301570 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-776d1537-bd70-4b30-a967-94cdff7b5e0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:09:37.417947 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 946/1867] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:09:37 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:09:37.422292 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5899d783-1c75-4fe1-98b0-75df5c062924 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:37.510734 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:09:37.542833 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5899d783-1c75-4fe1-98b0-75df5c062924 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:37.551381 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f799a7-a4db-48a0-a32a-3d4b259919f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:37.552205 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f799a7-a4db-48a0-a32a-3d4b259919f5 admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:37.552603 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f799a7-a4db-48a0-a32a-3d4b259919f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:37.552988 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f799a7-a4db-48a0-a32a-3d4b259919f5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:37.553846 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f799a7-a4db-48a0-a32a-3d4b259919f5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:37.555971 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f799a7-a4db-48a0-a32a-3d4b259919f5 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:37.595593 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 922/1868] 10.222.0.46 () {60 vars in 1152 bytes} [Thu May 26 15:09:37 2022] POST /identity/v3/projects => generated 377 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 15:09:37.599498 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776d1537-bd70-4b30-a967-94cdff7b5e0c None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:37.616532 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-776d1537-bd70-4b30-a967-94cdff7b5e0c None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:37.660462 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f799a7-a4db-48a0-a32a-3d4b259919f5 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:37.662480 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-f1f799a7-a4db-48a0-a32a-3d4b259919f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:09:37.763052 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea654d3-ddfc-427d-bf30-a9b0c6e1171c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:37.764742 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea654d3-ddfc-427d-bf30-a9b0c6e1171c admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:37.765190 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea654d3-ddfc-427d-bf30-a9b0c6e1171c admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:37.765504 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea654d3-ddfc-427d-bf30-a9b0c6e1171c admin admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:37.766808 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea654d3-ddfc-427d-bf30-a9b0c6e1171c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:37.770830 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea654d3-ddfc-427d-bf30-a9b0c6e1171c admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:37.790660 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f799a7-a4db-48a0-a32a-3d4b259919f5 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:37.796049 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97eddbcf-1e82-43a6-8a2b-5a0d486d4b68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:37.796443 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 947/1869] 10.222.0.46 () {60 vars in 1143 bytes} [Thu May 26 15:09:37 2022] POST /identity/v3/users => generated 289 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 15:09:37.800891 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97eddbcf-1e82-43a6-8a2b-5a0d486d4b68 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:37.801122 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97eddbcf-1e82-43a6-8a2b-5a0d486d4b68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:37.801624 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97eddbcf-1e82-43a6-8a2b-5a0d486d4b68 admin admin] PATH_INFO: `/v3/projects/27eba87c67f1479196cfe6b904db154f` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:37.803194 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97eddbcf-1e82-43a6-8a2b-5a0d486d4b68 admin admin] RBAC: Authorizing `identity:delete_project(project_id=27eba87c67f1479196cfe6b904db154f)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:37.804900 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea654d3-ddfc-427d-bf30-a9b0c6e1171c admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:37.805660 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97eddbcf-1e82-43a6-8a2b-5a0d486d4b68 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:37.824279 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea654d3-ddfc-427d-bf30-a9b0c6e1171c admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:37.830738 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca18bbfe-6315-400d-8f95-221106307098 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:37.831442 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca18bbfe-6315-400d-8f95-221106307098 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:37.831721 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca18bbfe-6315-400d-8f95-221106307098 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:37.832000 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca18bbfe-6315-400d-8f95-221106307098 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:37.832773 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca18bbfe-6315-400d-8f95-221106307098 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:37.834946 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca18bbfe-6315-400d-8f95-221106307098 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:37.851531 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 948/1870] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 15:09:37 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 15:09:37.854447 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca18bbfe-6315-400d-8f95-221106307098 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:37.865715 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca18bbfe-6315-400d-8f95-221106307098 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:37.879360 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-97eddbcf-1e82-43a6-8a2b-5a0d486d4b68 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '27eba87c67f1479196cfe6b904db154f', 'request_id': 'req-97eddbcf-1e82-43a6-8a2b-5a0d486d4b68'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:37.886758 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-97eddbcf-1e82-43a6-8a2b-5a0d486d4b68 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '27eba87c67f1479196cfe6b904db154f', 'request_id': 'req-97eddbcf-1e82-43a6-8a2b-5a0d486d4b68'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:37.896061 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-97eddbcf-1e82-43a6-8a2b-5a0d486d4b68 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '27eba87c67f1479196cfe6b904db154f', 'request_id': 'req-97eddbcf-1e82-43a6-8a2b-5a0d486d4b68'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:37.910241 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-97eddbcf-1e82-43a6-8a2b-5a0d486d4b68 admin admin] The token cache is being invalidate because project 27eba87c67f1479196cfe6b904db154f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 15:09:37.910630 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-97eddbcf-1e82-43a6-8a2b-5a0d486d4b68 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:37.911366 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-97eddbcf-1e82-43a6-8a2b-5a0d486d4b68 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:37.913041 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 923/1871] 10.222.0.46 () {58 vars in 1232 bytes} [Thu May 26 15:09:37 2022] DELETE /identity/v3/projects/27eba87c67f1479196cfe6b904db154f => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:09:37.916771 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97eddbcf-1e82-43a6-8a2b-5a0d486d4b68 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:37.917037 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:09:37.918318 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-97eddbcf-1e82-43a6-8a2b-5a0d486d4b68 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:09:37.938230 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4288db6-7bb6-4765-96a9-43023d441ccd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:37.939048 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4288db6-7bb6-4765-96a9-43023d441ccd admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:37.939327 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4288db6-7bb6-4765-96a9-43023d441ccd admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:37.939597 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4288db6-7bb6-4765-96a9-43023d441ccd admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:37.940323 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4288db6-7bb6-4765-96a9-43023d441ccd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:37.942026 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4288db6-7bb6-4765-96a9-43023d441ccd admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:37.952727 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 949/1872] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 15:09:37 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 15:09:37.953657 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4288db6-7bb6-4765-96a9-43023d441ccd admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:37.957242 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-a4288db6-7bb6-4765-96a9-43023d441ccd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:09:37.969411 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4288db6-7bb6-4765-96a9-43023d441ccd admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:37.974217 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fe4ed3-710f-48f4-b5e7-3a314f7dbd6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:37.975092 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fe4ed3-710f-48f4-b5e7-3a314f7dbd6e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:37.975373 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fe4ed3-710f-48f4-b5e7-3a314f7dbd6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:37.975651 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fe4ed3-710f-48f4-b5e7-3a314f7dbd6e admin admin] PATH_INFO: `/v3/users/933428759bd04f09b2023413757ec073` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:37.976858 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fe4ed3-710f-48f4-b5e7-3a314f7dbd6e admin admin] RBAC: Authorizing `identity:delete_user(user_id=933428759bd04f09b2023413757ec073)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:37.979007 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fe4ed3-710f-48f4-b5e7-3a314f7dbd6e admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:37.979503 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:09:37.986216 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97eddbcf-1e82-43a6-8a2b-5a0d486d4b68 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:37.992507 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7e89ac-fa46-4d78-9632-e1a6bc9799df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:37.993506 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7e89ac-fa46-4d78-9632-e1a6bc9799df admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:37.993802 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7e89ac-fa46-4d78-9632-e1a6bc9799df admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:37.994087 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7e89ac-fa46-4d78-9632-e1a6bc9799df admin admin] PATH_INFO: `/v3/projects/d698e430e3c84396a3c925c8732e9b84` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:37.995276 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7e89ac-fa46-4d78-9632-e1a6bc9799df admin admin] RBAC: Authorizing `identity:delete_project(project_id=d698e430e3c84396a3c925c8732e9b84)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:37.997262 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7e89ac-fa46-4d78-9632-e1a6bc9799df admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:38.031015 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-a5fe4ed3-710f-48f4-b5e7-3a314f7dbd6e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '933428759bd04f09b2023413757ec073'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:38.039574 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-a5fe4ed3-710f-48f4-b5e7-3a314f7dbd6e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '933428759bd04f09b2023413757ec073'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:38.040145 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-a5fe4ed3-710f-48f4-b5e7-3a314f7dbd6e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '933428759bd04f09b2023413757ec073'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:38.050984 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-7b7e89ac-fa46-4d78-9632-e1a6bc9799df admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd698e430e3c84396a3c925c8732e9b84', 'request_id': 'req-7b7e89ac-fa46-4d78-9632-e1a6bc9799df'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:38.053617 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-a5fe4ed3-710f-48f4-b5e7-3a314f7dbd6e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '933428759bd04f09b2023413757ec073'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:38.057672 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-7b7e89ac-fa46-4d78-9632-e1a6bc9799df admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd698e430e3c84396a3c925c8732e9b84', 'request_id': 'req-7b7e89ac-fa46-4d78-9632-e1a6bc9799df'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:38.060763 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-a5fe4ed3-710f-48f4-b5e7-3a314f7dbd6e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '933428759bd04f09b2023413757ec073'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:38.063213 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-7b7e89ac-fa46-4d78-9632-e1a6bc9799df admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd698e430e3c84396a3c925c8732e9b84', 'request_id': 'req-7b7e89ac-fa46-4d78-9632-e1a6bc9799df'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:38.067637 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 950/1873] 10.222.0.46 () {58 vars in 1223 bytes} [Thu May 26 15:09:37 2022] DELETE /identity/v3/users/933428759bd04f09b2023413757ec073 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:09:38.070841 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fe4ed3-710f-48f4-b5e7-3a314f7dbd6e admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:38.071271 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-a5fe4ed3-710f-48f4-b5e7-3a314f7dbd6e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:09:38.080017 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-7b7e89ac-fa46-4d78-9632-e1a6bc9799df admin admin] The token cache is being invalidate because project d698e430e3c84396a3c925c8732e9b84 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 15:09:38.080634 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-7b7e89ac-fa46-4d78-9632-e1a6bc9799df admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:38.081353 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-7b7e89ac-fa46-4d78-9632-e1a6bc9799df admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:38.083225 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 924/1874] 10.222.0.46 () {58 vars in 1232 bytes} [Thu May 26 15:09:37 2022] DELETE /identity/v3/projects/d698e430e3c84396a3c925c8732e9b84 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:09:38.086901 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7e89ac-fa46-4d78-9632-e1a6bc9799df admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:38.088314 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-7b7e89ac-fa46-4d78-9632-e1a6bc9799df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:09:38.162516 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fe4ed3-710f-48f4-b5e7-3a314f7dbd6e admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:38.167735 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b5cffe-e259-4f62-ad6c-ccc59652fcdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:38.168988 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b5cffe-e259-4f62-ad6c-ccc59652fcdc admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:38.169583 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b5cffe-e259-4f62-ad6c-ccc59652fcdc admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:38.170160 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b5cffe-e259-4f62-ad6c-ccc59652fcdc admin admin] PATH_INFO: `/v3/projects/b38f8fdcee1b49be9fcdf84534dc9b46/users/6fdcfb22d5be47dc9215d4cb0671efb4/roles/d4088d1845a24fbd9bd8975ce0b1d3d0` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:38.186529 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7e89ac-fa46-4d78-9632-e1a6bc9799df admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:38.193431 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b5cffe-e259-4f62-ad6c-ccc59652fcdc admin admin] RBAC: Authorizing `identity:create_grant(project_id=b38f8fdcee1b49be9fcdf84534dc9b46, user_id=6fdcfb22d5be47dc9215d4cb0671efb4, role_id=d4088d1845a24fbd9bd8975ce0b1d3d0)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:38.197525 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b5cffe-e259-4f62-ad6c-ccc59652fcdc admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:38.207481 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae753610-8b8a-41ca-b9f5-8e953ef522a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:38.208685 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae753610-8b8a-41ca-b9f5-8e953ef522a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:38.209956 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae753610-8b8a-41ca-b9f5-8e953ef522a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:38.210406 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae753610-8b8a-41ca-b9f5-8e953ef522a1 admin admin] PATH_INFO: `/v3/users/de10f500a75942d0b7486006b0f2e49f` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:38.214074 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae753610-8b8a-41ca-b9f5-8e953ef522a1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=de10f500a75942d0b7486006b0f2e49f)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:38.216581 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae753610-8b8a-41ca-b9f5-8e953ef522a1 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:38.249552 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 951/1875] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 15:09:38 2022] PUT /identity/v3/projects/b38f8fdcee1b49be9fcdf84534dc9b46/users/6fdcfb22d5be47dc9215d4cb0671efb4/roles/d4088d1845a24fbd9bd8975ce0b1d3d0 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:09:38.251311 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2be900-8aba-4059-be1d-a43e611893ad None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:38.251607 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2be900-8aba-4059-be1d-a43e611893ad None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:38.251890 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2be900-8aba-4059-be1d-a43e611893ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:38.295332 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-ef2be900-8aba-4059-be1d-a43e611893ad None None] MFA Rules not processed for user `6fdcfb22d5be47dc9215d4cb0671efb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:09:38.296661 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-ef2be900-8aba-4059-be1d-a43e611893ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:09:38.352959 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-ae753610-8b8a-41ca-b9f5-8e953ef522a1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'de10f500a75942d0b7486006b0f2e49f'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:38.366725 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:09:38.381634 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-ae753610-8b8a-41ca-b9f5-8e953ef522a1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'de10f500a75942d0b7486006b0f2e49f'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:38.382472 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-ae753610-8b8a-41ca-b9f5-8e953ef522a1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'de10f500a75942d0b7486006b0f2e49f'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:38.413923 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-ae753610-8b8a-41ca-b9f5-8e953ef522a1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'de10f500a75942d0b7486006b0f2e49f'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:38.436144 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-ae753610-8b8a-41ca-b9f5-8e953ef522a1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'de10f500a75942d0b7486006b0f2e49f'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:38.447702 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 925/1876] 10.222.0.46 () {58 vars in 1223 bytes} [Thu May 26 15:09:38 2022] DELETE /identity/v3/users/de10f500a75942d0b7486006b0f2e49f => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 15:09:38.489216 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 952/1877] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:09:38 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:09:38.494249 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae753610-8b8a-41ca-b9f5-8e953ef522a1 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:38.495536 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-ae753610-8b8a-41ca-b9f5-8e953ef522a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:09:38.515383 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:09:38.666062 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae753610-8b8a-41ca-b9f5-8e953ef522a1 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:38.677260 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2604a40a-2c6e-4179-960a-30a910d84147 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:38.678521 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2604a40a-2c6e-4179-960a-30a910d84147 admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:38.679256 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2604a40a-2c6e-4179-960a-30a910d84147 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:38.679699 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2604a40a-2c6e-4179-960a-30a910d84147 admin admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:38.681262 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2604a40a-2c6e-4179-960a-30a910d84147 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:38.685481 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2604a40a-2c6e-4179-960a-30a910d84147 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:38.734208 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2be900-8aba-4059-be1d-a43e611893ad None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:38.737096 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-ef2be900-8aba-4059-be1d-a43e611893ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:09:38.770332 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 926/1878] 10.222.0.46 () {60 vars in 1152 bytes} [Thu May 26 15:09:38 2022] POST /identity/v3/projects => generated 377 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 15:09:38.775268 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2604a40a-2c6e-4179-960a-30a910d84147 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:38.891849 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2604a40a-2c6e-4179-960a-30a910d84147 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:38.898218 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22982bcd-6bcb-4abc-aee2-0d97a053354a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:38.898978 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22982bcd-6bcb-4abc-aee2-0d97a053354a admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:38.899267 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22982bcd-6bcb-4abc-aee2-0d97a053354a admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:38.899550 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22982bcd-6bcb-4abc-aee2-0d97a053354a admin admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:38.900259 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22982bcd-6bcb-4abc-aee2-0d97a053354a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:38.902487 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22982bcd-6bcb-4abc-aee2-0d97a053354a admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:38.962619 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2be900-8aba-4059-be1d-a43e611893ad None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:38.971753 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 927/1879] 10.222.0.46 () {60 vars in 1143 bytes} [Thu May 26 15:09:38 2022] POST /identity/v3/users => generated 295 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 15:09:38.980005 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22982bcd-6bcb-4abc-aee2-0d97a053354a admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:38.986142 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ad9084-a946-47ae-bf53-2f7b107a23d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:38.991197 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ad9084-a946-47ae-bf53-2f7b107a23d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:38.991794 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ad9084-a946-47ae-bf53-2f7b107a23d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:38.993284 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ad9084-a946-47ae-bf53-2f7b107a23d4 admin admin] PATH_INFO: `/v3/users/8afca1c17da04182ad09bf43a8a4284a` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:38.998832 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ad9084-a946-47ae-bf53-2f7b107a23d4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8afca1c17da04182ad09bf43a8a4284a)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:39.005507 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ad9084-a946-47ae-bf53-2f7b107a23d4 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:39.017987 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22982bcd-6bcb-4abc-aee2-0d97a053354a admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:39.033762 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9fe3df-d942-4787-aca2-9b0ba6ded0b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:39.034837 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9fe3df-d942-4787-aca2-9b0ba6ded0b7 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:39.035233 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9fe3df-d942-4787-aca2-9b0ba6ded0b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:39.035669 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9fe3df-d942-4787-aca2-9b0ba6ded0b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:39.036548 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9fe3df-d942-4787-aca2-9b0ba6ded0b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:39.042025 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d9fe3df-d942-4787-aca2-9b0ba6ded0b7 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:39.065094 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 928/1880] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 15:09:38 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 15:09:39.077075 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9fe3df-d942-4787-aca2-9b0ba6ded0b7 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:39.077931 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-5d9fe3df-d942-4787-aca2-9b0ba6ded0b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:09:39.093624 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9fe3df-d942-4787-aca2-9b0ba6ded0b7 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:39.100137 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34621301-a85f-4179-91ca-a266ed0733cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:39.102001 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34621301-a85f-4179-91ca-a266ed0733cb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:39.102395 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34621301-a85f-4179-91ca-a266ed0733cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:39.102679 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34621301-a85f-4179-91ca-a266ed0733cb admin admin] PATH_INFO: `/v3/projects/6fb676ab0c6041769602aeaa37855a3c` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:39.104185 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34621301-a85f-4179-91ca-a266ed0733cb admin admin] RBAC: Authorizing `identity:delete_project(project_id=6fb676ab0c6041769602aeaa37855a3c)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:39.106677 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-50ad9084-a946-47ae-bf53-2f7b107a23d4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8afca1c17da04182ad09bf43a8a4284a'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:39.106816 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34621301-a85f-4179-91ca-a266ed0733cb admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:39.116280 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-50ad9084-a946-47ae-bf53-2f7b107a23d4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8afca1c17da04182ad09bf43a8a4284a'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:39.117247 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-50ad9084-a946-47ae-bf53-2f7b107a23d4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8afca1c17da04182ad09bf43a8a4284a'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:39.138510 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-50ad9084-a946-47ae-bf53-2f7b107a23d4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8afca1c17da04182ad09bf43a8a4284a'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:39.150326 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-50ad9084-a946-47ae-bf53-2f7b107a23d4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8afca1c17da04182ad09bf43a8a4284a'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:39.158915 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 953/1881] 10.222.0.46 () {58 vars in 1223 bytes} [Thu May 26 15:09:38 2022] DELETE /identity/v3/users/8afca1c17da04182ad09bf43a8a4284a => generated 0 bytes in 425 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:09:39.159871 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ad9084-a946-47ae-bf53-2f7b107a23d4 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:39.161642 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-50ad9084-a946-47ae-bf53-2f7b107a23d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:09:39.171201 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-34621301-a85f-4179-91ca-a266ed0733cb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6fb676ab0c6041769602aeaa37855a3c', 'request_id': 'req-34621301-a85f-4179-91ca-a266ed0733cb'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:39.180853 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-34621301-a85f-4179-91ca-a266ed0733cb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6fb676ab0c6041769602aeaa37855a3c', 'request_id': 'req-34621301-a85f-4179-91ca-a266ed0733cb'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:39.190038 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-34621301-a85f-4179-91ca-a266ed0733cb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6fb676ab0c6041769602aeaa37855a3c', 'request_id': 'req-34621301-a85f-4179-91ca-a266ed0733cb'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:39.192472 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=90,keystone:INSERT=4,keystone:DELETE=6 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:09:39.226394 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:09:39.247881 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-34621301-a85f-4179-91ca-a266ed0733cb admin admin] The token cache is being invalidate because project 6fb676ab0c6041769602aeaa37855a3c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 15:09:39.248190 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-34621301-a85f-4179-91ca-a266ed0733cb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:39.251116 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-34621301-a85f-4179-91ca-a266ed0733cb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:39.254345 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 929/1882] 10.222.0.46 () {58 vars in 1232 bytes} [Thu May 26 15:09:39 2022] DELETE /identity/v3/projects/6fb676ab0c6041769602aeaa37855a3c => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:09:39.256297 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f41caf3-223e-401f-82ce-61fe15c4dc52 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:39.256594 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f41caf3-223e-401f-82ce-61fe15c4dc52 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:39.256877 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f41caf3-223e-401f-82ce-61fe15c4dc52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:39.278688 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ad9084-a946-47ae-bf53-2f7b107a23d4 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:39.291850 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960f16c8-e41f-4ccf-a732-af2bc60c452c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:39.292959 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960f16c8-e41f-4ccf-a732-af2bc60c452c admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:39.293293 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960f16c8-e41f-4ccf-a732-af2bc60c452c admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:39.293721 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960f16c8-e41f-4ccf-a732-af2bc60c452c admin admin] PATH_INFO: `/v3/projects/92f36451fefc449cbea3b78dfa0c7c26/users/69016d2ad4db4bb78570b588a0a8dab0/roles/a2e551773cbd4df084020676865b8222` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:39.302111 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-6f41caf3-223e-401f-82ce-61fe15c4dc52 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:09:39.308175 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-6f41caf3-223e-401f-82ce-61fe15c4dc52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:09:39.326107 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 930/1883] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:09:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:09:39.331995 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ab487f-91a0-4a61-9321-d3bb520b26f1 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:39.332291 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ab487f-91a0-4a61-9321-d3bb520b26f1 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:39.332627 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ab487f-91a0-4a61-9321-d3bb520b26f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:39.334540 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960f16c8-e41f-4ccf-a732-af2bc60c452c admin admin] RBAC: Authorizing `identity:create_grant(project_id=92f36451fefc449cbea3b78dfa0c7c26, user_id=69016d2ad4db4bb78570b588a0a8dab0, role_id=a2e551773cbd4df084020676865b8222)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:39.337326 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960f16c8-e41f-4ccf-a732-af2bc60c452c admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:39.358029 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-f2ab487f-91a0-4a61-9321-d3bb520b26f1 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:09:39.359578 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-f2ab487f-91a0-4a61-9321-d3bb520b26f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:09:39.360179 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 954/1884] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 15:09:39 2022] PUT /identity/v3/projects/92f36451fefc449cbea3b78dfa0c7c26/users/69016d2ad4db4bb78570b588a0a8dab0/roles/a2e551773cbd4df084020676865b8222 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:09:39.364678 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960f16c8-e41f-4ccf-a732-af2bc60c452c admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:39.367184 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-960f16c8-e41f-4ccf-a732-af2bc60c452c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:09:39.523572 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 931/1885] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:09:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:09:39.524193 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ab487f-91a0-4a61-9321-d3bb520b26f1 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:39.527993 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-f2ab487f-91a0-4a61-9321-d3bb520b26f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:09:39.577055 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960f16c8-e41f-4ccf-a732-af2bc60c452c admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:39.599127 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506b0f20-aec4-46ef-8945-47b2b363c534 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:39.599835 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506b0f20-aec4-46ef-8945-47b2b363c534 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:39.600121 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506b0f20-aec4-46ef-8945-47b2b363c534 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:39.600401 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506b0f20-aec4-46ef-8945-47b2b363c534 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:39.601816 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506b0f20-aec4-46ef-8945-47b2b363c534 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:39.602850 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506b0f20-aec4-46ef-8945-47b2b363c534 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:39.635914 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 955/1886] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 15:09:39 2022] GET /identity/v3/roles => generated 1420 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 15:09:39.639859 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1082b1c4-4aff-496b-9742-e094c9f02be6 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:39.640238 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1082b1c4-4aff-496b-9742-e094c9f02be6 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:39.640531 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1082b1c4-4aff-496b-9742-e094c9f02be6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:39.692850 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-1082b1c4-4aff-496b-9742-e094c9f02be6 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:09:39.694759 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-1082b1c4-4aff-496b-9742-e094c9f02be6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:09:39.705303 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 956/1887] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:09:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:09:39.707268 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1082b1c4-4aff-496b-9742-e094c9f02be6 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:39.714162 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ab487f-91a0-4a61-9321-d3bb520b26f1 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:39.723827 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-989cd53a-b0f1-41fe-95dc-dd70ff711abc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0156a2f6f3794f479e76b85bdbbe5c26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:39.724587 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989cd53a-b0f1-41fe-95dc-dd70ff711abc service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:39.724878 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989cd53a-b0f1-41fe-95dc-dd70ff711abc service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:39.725168 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-989cd53a-b0f1-41fe-95dc-dd70ff711abc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:39.730580 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1082b1c4-4aff-496b-9742-e094c9f02be6 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:39.738551 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924bd1a7-7801-4539-99b0-1eb2fcba307a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:39.739574 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924bd1a7-7801-4539-99b0-1eb2fcba307a admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:39.739917 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924bd1a7-7801-4539-99b0-1eb2fcba307a admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:39.740259 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924bd1a7-7801-4539-99b0-1eb2fcba307a admin admin] PATH_INFO: `/v3/projects/92f36451fefc449cbea3b78dfa0c7c26/users/69016d2ad4db4bb78570b588a0a8dab0/roles/a2e551773cbd4df084020676865b8222` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:39.743838 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924bd1a7-7801-4539-99b0-1eb2fcba307a admin admin] RBAC: Authorizing `identity:create_grant(project_id=92f36451fefc449cbea3b78dfa0c7c26, user_id=69016d2ad4db4bb78570b588a0a8dab0, role_id=a2e551773cbd4df084020676865b8222)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:39.746559 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989cd53a-b0f1-41fe-95dc-dd70ff711abc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:39.748606 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-924bd1a7-7801-4539-99b0-1eb2fcba307a admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:39.750931 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-989cd53a-b0f1-41fe-95dc-dd70ff711abc service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:39.765534 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 957/1888] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 15:09:39 2022] PUT /identity/v3/projects/92f36451fefc449cbea3b78dfa0c7c26/users/69016d2ad4db4bb78570b588a0a8dab0/roles/a2e551773cbd4df084020676865b8222 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:09:39.766650 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924bd1a7-7801-4539-99b0-1eb2fcba307a admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:39.773527 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 932/1889] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:09:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 250 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 15:09:39.778962 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01896bab-5ef7-485d-8a7a-eaeeb237b285 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:39.779270 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01896bab-5ef7-485d-8a7a-eaeeb237b285 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:39.779531 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01896bab-5ef7-485d-8a7a-eaeeb237b285 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:39.831201 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-01896bab-5ef7-485d-8a7a-eaeeb237b285 None None] MFA Rules not processed for user `69016d2ad4db4bb78570b588a0a8dab0`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:09:39.832642 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-01896bab-5ef7-485d-8a7a-eaeeb237b285 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:09:39.873288 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924bd1a7-7801-4539-99b0-1eb2fcba307a admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:39.890352 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5b1e93-41bf-480f-bded-2dc9c5b05918 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:39.891188 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5b1e93-41bf-480f-bded-2dc9c5b05918 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:39.891540 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5b1e93-41bf-480f-bded-2dc9c5b05918 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:39.892178 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5b1e93-41bf-480f-bded-2dc9c5b05918 admin admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:39.893328 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5b1e93-41bf-480f-bded-2dc9c5b05918 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 15:09:39.893679 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5b1e93-41bf-480f-bded-2dc9c5b05918 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:39.897667 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5b1e93-41bf-480f-bded-2dc9c5b05918 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:39.913237 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 958/1890] 10.222.0.46 () {58 vars in 1152 bytes} [Thu May 26 15:09:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 15:09:39.914421 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5b1e93-41bf-480f-bded-2dc9c5b05918 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:39.916639 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-db5b1e93-41bf-480f-bded-2dc9c5b05918 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:09:39.933530 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5b1e93-41bf-480f-bded-2dc9c5b05918 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:39.942084 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a92029-c531-4b32-9b63-a99ccabcaeeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:39.943256 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a92029-c531-4b32-9b63-a99ccabcaeeb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:39.943647 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a92029-c531-4b32-9b63-a99ccabcaeeb admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:39.944074 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a92029-c531-4b32-9b63-a99ccabcaeeb admin admin] PATH_INFO: `/v3/projects/67808d9e84714011b25197dbeeeb989d` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:39.945667 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a92029-c531-4b32-9b63-a99ccabcaeeb admin admin] RBAC: Authorizing `identity:delete_project(project_id=67808d9e84714011b25197dbeeeb989d)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:39.948089 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a92029-c531-4b32-9b63-a99ccabcaeeb admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:39.995449 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=96,keystone:INSERT=2,keystone:DELETE=2 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:09:40.003774 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 933/1891] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:09:39 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:09:40.007610 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01896bab-5ef7-485d-8a7a-eaeeb237b285 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:40.035831 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-81a92029-c531-4b32-9b63-a99ccabcaeeb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '67808d9e84714011b25197dbeeeb989d', 'request_id': 'req-81a92029-c531-4b32-9b63-a99ccabcaeeb'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:40.037760 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01896bab-5ef7-485d-8a7a-eaeeb237b285 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:40.050205 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-81a92029-c531-4b32-9b63-a99ccabcaeeb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '67808d9e84714011b25197dbeeeb989d', 'request_id': 'req-81a92029-c531-4b32-9b63-a99ccabcaeeb'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:40.054651 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:INSERT=1,keystone:DELETE=1,keystone:SELECT=98 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:09:40.074402 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-81a92029-c531-4b32-9b63-a99ccabcaeeb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '67808d9e84714011b25197dbeeeb989d', 'request_id': 'req-81a92029-c531-4b32-9b63-a99ccabcaeeb'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:40.122046 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-81a92029-c531-4b32-9b63-a99ccabcaeeb admin admin] The token cache is being invalidate because project 67808d9e84714011b25197dbeeeb989d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 15:09:40.122564 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-81a92029-c531-4b32-9b63-a99ccabcaeeb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:40.124048 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-81a92029-c531-4b32-9b63-a99ccabcaeeb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:40.128078 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 959/1892] 10.222.0.46 () {58 vars in 1232 bytes} [Thu May 26 15:09:39 2022] DELETE /identity/v3/projects/67808d9e84714011b25197dbeeeb989d => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:09:40.129045 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a92029-c531-4b32-9b63-a99ccabcaeeb admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:40.133082 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-81a92029-c531-4b32-9b63-a99ccabcaeeb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:09:40.153435 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a826f6e-a5a4-4f6d-a961-92626a4f61fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:40.155680 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a826f6e-a5a4-4f6d-a961-92626a4f61fe admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:40.156192 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a826f6e-a5a4-4f6d-a961-92626a4f61fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:40.156686 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a826f6e-a5a4-4f6d-a961-92626a4f61fe admin admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:40.157769 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a826f6e-a5a4-4f6d-a961-92626a4f61fe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:40.163149 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a826f6e-a5a4-4f6d-a961-92626a4f61fe admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:40.216250 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 934/1893] 10.222.0.46 () {60 vars in 1152 bytes} [Thu May 26 15:09:40 2022] POST /identity/v3/projects => generated 383 bytes in 208 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 15:09:40.216250 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889350b4-072c-446b-bbca-a339a4c469c0 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:40.216250 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889350b4-072c-446b-bbca-a339a4c469c0 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:40.216250 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889350b4-072c-446b-bbca-a339a4c469c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:40.250053 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-889350b4-072c-446b-bbca-a339a4c469c0 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:09:40.251725 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-889350b4-072c-446b-bbca-a339a4c469c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:09:40.279493 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a92029-c531-4b32-9b63-a99ccabcaeeb admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:40.348965 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 935/1894] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:09:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:09:40.350668 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889350b4-072c-446b-bbca-a339a4c469c0 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:40.352290 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-889350b4-072c-446b-bbca-a339a4c469c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:09:40.366471 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889350b4-072c-446b-bbca-a339a4c469c0 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:40.378573 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3686f6b5-8da5-46fa-9da0-0fa0873ae327 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:40.378573 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3686f6b5-8da5-46fa-9da0-0fa0873ae327 admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:40.378573 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3686f6b5-8da5-46fa-9da0-0fa0873ae327 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:40.378573 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3686f6b5-8da5-46fa-9da0-0fa0873ae327 admin admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:40.378993 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3686f6b5-8da5-46fa-9da0-0fa0873ae327 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:40.378993 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3686f6b5-8da5-46fa-9da0-0fa0873ae327 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:40.381136 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac46e151-1d81-42f3-8000-9d161319bb1a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5b5b35614a547d783e6e77ddb8c1c03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:40.381841 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac46e151-1d81-42f3-8000-9d161319bb1a service cinder] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:40.382220 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac46e151-1d81-42f3-8000-9d161319bb1a service cinder] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:40.382411 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac46e151-1d81-42f3-8000-9d161319bb1a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:40.384375 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-ac46e151-1d81-42f3-8000-9d161319bb1a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:09:40.400895 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 936/1895] 10.222.0.46 () {60 vars in 1143 bytes} [Thu May 26 15:09:40 2022] POST /identity/v3/users => generated 292 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 15:09:40.402449 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3686f6b5-8da5-46fa-9da0-0fa0873ae327 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:40.406840 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-3686f6b5-8da5-46fa-9da0-0fa0873ae327 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:09:40.423054 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3686f6b5-8da5-46fa-9da0-0fa0873ae327 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:40.428965 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770aeba0-9d43-4544-abbd-1a56430068e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:40.430218 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770aeba0-9d43-4544-abbd-1a56430068e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:40.430671 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770aeba0-9d43-4544-abbd-1a56430068e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:40.431124 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770aeba0-9d43-4544-abbd-1a56430068e4 admin admin] PATH_INFO: `/v3/projects/76b9a456767c4aa99d73aee5a20a6172` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:40.433294 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770aeba0-9d43-4544-abbd-1a56430068e4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=76b9a456767c4aa99d73aee5a20a6172)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:40.436059 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770aeba0-9d43-4544-abbd-1a56430068e4 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:40.460736 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac46e151-1d81-42f3-8000-9d161319bb1a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:40.462594 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac46e151-1d81-42f3-8000-9d161319bb1a service cinder] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:40.475656 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 960/1896] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:09:40 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 15:09:40.477596 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51badb8a-42a7-4f76-adc7-6135a0a395b2 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:40.477967 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51badb8a-42a7-4f76-adc7-6135a0a395b2 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:40.478330 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51badb8a-42a7-4f76-adc7-6135a0a395b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:40.500738 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-770aeba0-9d43-4544-abbd-1a56430068e4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '76b9a456767c4aa99d73aee5a20a6172', 'request_id': 'req-770aeba0-9d43-4544-abbd-1a56430068e4'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:40.503337 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-51badb8a-42a7-4f76-adc7-6135a0a395b2 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:09:40.506190 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-51badb8a-42a7-4f76-adc7-6135a0a395b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:09:40.508983 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-770aeba0-9d43-4544-abbd-1a56430068e4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '76b9a456767c4aa99d73aee5a20a6172', 'request_id': 'req-770aeba0-9d43-4544-abbd-1a56430068e4'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:40.516756 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-770aeba0-9d43-4544-abbd-1a56430068e4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '76b9a456767c4aa99d73aee5a20a6172', 'request_id': 'req-770aeba0-9d43-4544-abbd-1a56430068e4'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:40.534092 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-770aeba0-9d43-4544-abbd-1a56430068e4 admin admin] The token cache is being invalidate because project 76b9a456767c4aa99d73aee5a20a6172 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 15:09:40.534448 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-770aeba0-9d43-4544-abbd-1a56430068e4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:40.535787 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-770aeba0-9d43-4544-abbd-1a56430068e4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:40.537705 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:09:40.538908 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 937/1897] 10.222.0.46 () {58 vars in 1232 bytes} [Thu May 26 15:09:40 2022] DELETE /identity/v3/projects/76b9a456767c4aa99d73aee5a20a6172 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:09:40.540336 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770aeba0-9d43-4544-abbd-1a56430068e4 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:40.541942 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-770aeba0-9d43-4544-abbd-1a56430068e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:09:40.552251 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:09:40.616902 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 961/1898] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:09:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:09:40.623283 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51badb8a-42a7-4f76-adc7-6135a0a395b2 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:40.628156 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-51badb8a-42a7-4f76-adc7-6135a0a395b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:09:40.649540 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51badb8a-42a7-4f76-adc7-6135a0a395b2 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:40.659965 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42acceea-0d3a-4466-b4d0-8b661738d5dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:40.661618 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42acceea-0d3a-4466-b4d0-8b661738d5dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:40.661618 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42acceea-0d3a-4466-b4d0-8b661738d5dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:40.662300 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42acceea-0d3a-4466-b4d0-8b661738d5dd admin admin] PATH_INFO: `/v3/users/efb0e1c8fd8a4b1e81826a029fbb0da3` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:40.664149 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42acceea-0d3a-4466-b4d0-8b661738d5dd admin admin] RBAC: Authorizing `identity:delete_user(user_id=efb0e1c8fd8a4b1e81826a029fbb0da3)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:40.667992 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42acceea-0d3a-4466-b4d0-8b661738d5dd admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:40.673981 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770aeba0-9d43-4544-abbd-1a56430068e4 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:40.682867 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c795e44-d7f4-44e1-aa43-7902ba7f8b21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:40.685100 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c795e44-d7f4-44e1-aa43-7902ba7f8b21 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:40.685595 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c795e44-d7f4-44e1-aa43-7902ba7f8b21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:40.685884 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c795e44-d7f4-44e1-aa43-7902ba7f8b21 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:40.686810 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c795e44-d7f4-44e1-aa43-7902ba7f8b21 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:40.689805 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c795e44-d7f4-44e1-aa43-7902ba7f8b21 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:40.711649 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 938/1899] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 15:09:40 2022] GET /identity/v3/roles => generated 1420 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 15:09:40.712595 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c795e44-d7f4-44e1-aa43-7902ba7f8b21 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:40.729272 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c795e44-d7f4-44e1-aa43-7902ba7f8b21 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:40.741113 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656eb697-9778-4a72-a078-7e728946993e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:40.741838 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656eb697-9778-4a72-a078-7e728946993e admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:40.742135 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656eb697-9778-4a72-a078-7e728946993e admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:40.742416 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656eb697-9778-4a72-a078-7e728946993e admin admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:40.743201 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656eb697-9778-4a72-a078-7e728946993e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 15:09:40.743461 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656eb697-9778-4a72-a078-7e728946993e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:40.745125 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656eb697-9778-4a72-a078-7e728946993e admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:40.755766 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-42acceea-0d3a-4466-b4d0-8b661738d5dd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'efb0e1c8fd8a4b1e81826a029fbb0da3'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:40.764333 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 939/1900] 10.222.0.46 () {58 vars in 1152 bytes} [Thu May 26 15:09:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 15:09:40.766604 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656eb697-9778-4a72-a078-7e728946993e admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:40.769156 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-42acceea-0d3a-4466-b4d0-8b661738d5dd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'efb0e1c8fd8a4b1e81826a029fbb0da3'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:40.769842 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-42acceea-0d3a-4466-b4d0-8b661738d5dd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'efb0e1c8fd8a4b1e81826a029fbb0da3'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:40.772704 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-656eb697-9778-4a72-a078-7e728946993e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:09:40.786469 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656eb697-9778-4a72-a078-7e728946993e admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:40.797327 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7220eac9-5212-4e12-af53-e38d1a521882 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:40.798350 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7220eac9-5212-4e12-af53-e38d1a521882 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:40.798643 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7220eac9-5212-4e12-af53-e38d1a521882 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:40.799679 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-42acceea-0d3a-4466-b4d0-8b661738d5dd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'efb0e1c8fd8a4b1e81826a029fbb0da3'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:40.799862 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7220eac9-5212-4e12-af53-e38d1a521882 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:40.800850 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7220eac9-5212-4e12-af53-e38d1a521882 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:40.802475 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7220eac9-5212-4e12-af53-e38d1a521882 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:40.811062 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-42acceea-0d3a-4466-b4d0-8b661738d5dd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'efb0e1c8fd8a4b1e81826a029fbb0da3'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:40.818559 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 940/1901] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 15:09:40 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 15:09:40.819509 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7220eac9-5212-4e12-af53-e38d1a521882 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:40.822503 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-7220eac9-5212-4e12-af53-e38d1a521882 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:09:40.827011 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 962/1902] 10.222.0.46 () {58 vars in 1223 bytes} [Thu May 26 15:09:40 2022] DELETE /identity/v3/users/efb0e1c8fd8a4b1e81826a029fbb0da3 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:09:40.827866 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42acceea-0d3a-4466-b4d0-8b661738d5dd admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:40.890319 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7220eac9-5212-4e12-af53-e38d1a521882 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:40.969283 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42acceea-0d3a-4466-b4d0-8b661738d5dd admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:40.980192 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42de10f-58c2-4b8c-be17-a88a559e9b8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:40.980520 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42de10f-58c2-4b8c-be17-a88a559e9b8b admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:40.981339 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42de10f-58c2-4b8c-be17-a88a559e9b8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:40.981849 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42de10f-58c2-4b8c-be17-a88a559e9b8b admin admin] PATH_INFO: `/v3/projects/bdc9f8ca84bb4d33b2c7f76ab47f58d0/users/fd62eb0876fa4a869eb1ec531a3675be/roles/d4088d1845a24fbd9bd8975ce0b1d3d0` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:41.023614 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779ee999-953f-4a38-9781-ba6d8b254ebf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:41.024818 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779ee999-953f-4a38-9781-ba6d8b254ebf admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:41.025391 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779ee999-953f-4a38-9781-ba6d8b254ebf admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:41.025520 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42de10f-58c2-4b8c-be17-a88a559e9b8b admin admin] RBAC: Authorizing `identity:create_grant(project_id=bdc9f8ca84bb4d33b2c7f76ab47f58d0, user_id=fd62eb0876fa4a869eb1ec531a3675be, role_id=d4088d1845a24fbd9bd8975ce0b1d3d0)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:41.025983 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779ee999-953f-4a38-9781-ba6d8b254ebf admin admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:41.026837 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779ee999-953f-4a38-9781-ba6d8b254ebf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:41.029812 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779ee999-953f-4a38-9781-ba6d8b254ebf admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:41.033626 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42de10f-58c2-4b8c-be17-a88a559e9b8b admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:41.054674 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 963/1903] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 15:09:40 2022] PUT /identity/v3/projects/bdc9f8ca84bb4d33b2c7f76ab47f58d0/users/fd62eb0876fa4a869eb1ec531a3675be/roles/d4088d1845a24fbd9bd8975ce0b1d3d0 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:09:41.069010 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adbc1b9-f799-4a61-9518-cebec0369e05 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:41.069393 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adbc1b9-f799-4a61-9518-cebec0369e05 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:41.069926 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adbc1b9-f799-4a61-9518-cebec0369e05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:41.107872 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 941/1904] 10.222.0.46 () {60 vars in 1152 bytes} [Thu May 26 15:09:40 2022] POST /identity/v3/projects => generated 381 bytes in 285 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 15:09:41.109544 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779ee999-953f-4a38-9781-ba6d8b254ebf admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:41.128865 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-0adbc1b9-f799-4a61-9518-cebec0369e05 None None] MFA Rules not processed for user `fd62eb0876fa4a869eb1ec531a3675be`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:09:41.133141 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-0adbc1b9-f799-4a61-9518-cebec0369e05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:09:41.147185 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:09:41.226720 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779ee999-953f-4a38-9781-ba6d8b254ebf admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:41.235097 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6365a3-47b1-4343-aa1c-6703f9bab788 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:41.236122 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6365a3-47b1-4343-aa1c-6703f9bab788 admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:41.236629 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6365a3-47b1-4343-aa1c-6703f9bab788 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:41.237134 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b6365a3-47b1-4343-aa1c-6703f9bab788 admin admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:41.238093 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6365a3-47b1-4343-aa1c-6703f9bab788 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:41.254360 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b6365a3-47b1-4343-aa1c-6703f9bab788 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:41.278449 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:09:41.299293 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 964/1905] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:09:41 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:09:41.300008 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adbc1b9-f799-4a61-9518-cebec0369e05 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:41.305855 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-0adbc1b9-f799-4a61-9518-cebec0369e05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:09:41.319848 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 942/1906] 10.222.0.46 () {60 vars in 1143 bytes} [Thu May 26 15:09:41 2022] POST /identity/v3/users => generated 291 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 15:09:41.321196 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6365a3-47b1-4343-aa1c-6703f9bab788 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:41.334221 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adbc1b9-f799-4a61-9518-cebec0369e05 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:41.334907 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b6365a3-47b1-4343-aa1c-6703f9bab788 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:41.342413 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b5f5eb-cb4d-4b84-b030-8214b7f6e9e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:41.343722 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b5f5eb-cb4d-4b84-b030-8214b7f6e9e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:41.344773 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b5f5eb-cb4d-4b84-b030-8214b7f6e9e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:41.345201 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b5f5eb-cb4d-4b84-b030-8214b7f6e9e8 admin admin] PATH_INFO: `/v3/projects/8834518f444848799f7a09e24fe43f18` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:41.346867 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b5f5eb-cb4d-4b84-b030-8214b7f6e9e8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8834518f444848799f7a09e24fe43f18)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:41.349717 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b5f5eb-cb4d-4b84-b030-8214b7f6e9e8 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:41.358275 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3432558-8a14-4d69-a245-15b84fef9085 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:41.359402 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3432558-8a14-4d69-a245-15b84fef9085 admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:41.360985 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3432558-8a14-4d69-a245-15b84fef9085 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:41.361548 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3432558-8a14-4d69-a245-15b84fef9085 admin admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:41.362852 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3432558-8a14-4d69-a245-15b84fef9085 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:41.365803 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3432558-8a14-4d69-a245-15b84fef9085 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:41.413311 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 943/1907] 10.222.0.46 () {60 vars in 1152 bytes} [Thu May 26 15:09:41 2022] POST /identity/v3/projects => generated 381 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 15:09:41.413311 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3432558-8a14-4d69-a245-15b84fef9085 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:41.431036 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-78b5f5eb-cb4d-4b84-b030-8214b7f6e9e8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8834518f444848799f7a09e24fe43f18', 'request_id': 'req-78b5f5eb-cb4d-4b84-b030-8214b7f6e9e8'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:41.445851 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-78b5f5eb-cb4d-4b84-b030-8214b7f6e9e8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8834518f444848799f7a09e24fe43f18', 'request_id': 'req-78b5f5eb-cb4d-4b84-b030-8214b7f6e9e8'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:41.473882 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-78b5f5eb-cb4d-4b84-b030-8214b7f6e9e8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8834518f444848799f7a09e24fe43f18', 'request_id': 'req-78b5f5eb-cb4d-4b84-b030-8214b7f6e9e8'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:41.490957 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-78b5f5eb-cb4d-4b84-b030-8214b7f6e9e8 admin admin] The token cache is being invalidate because project 8834518f444848799f7a09e24fe43f18 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 15:09:41.491250 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-78b5f5eb-cb4d-4b84-b030-8214b7f6e9e8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:41.491844 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-78b5f5eb-cb4d-4b84-b030-8214b7f6e9e8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:41.496625 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 965/1908] 10.222.0.46 () {58 vars in 1232 bytes} [Thu May 26 15:09:41 2022] DELETE /identity/v3/projects/8834518f444848799f7a09e24fe43f18 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:09:41.496625 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b5f5eb-cb4d-4b84-b030-8214b7f6e9e8 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:41.496625 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-78b5f5eb-cb4d-4b84-b030-8214b7f6e9e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:09:41.517947 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3432558-8a14-4d69-a245-15b84fef9085 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:41.526310 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a10641c-fd17-4c73-a92e-511a3cf5278f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:41.527179 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a10641c-fd17-4c73-a92e-511a3cf5278f admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:41.527511 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a10641c-fd17-4c73-a92e-511a3cf5278f admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:41.528837 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a10641c-fd17-4c73-a92e-511a3cf5278f admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:41.529743 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a10641c-fd17-4c73-a92e-511a3cf5278f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:41.532965 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a10641c-fd17-4c73-a92e-511a3cf5278f admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:41.551497 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 944/1909] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 15:09:41 2022] GET /identity/v3/roles => generated 1420 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 15:09:41.551497 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc25898-fac3-4ac8-abac-fed00d81bbc0 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:41.551497 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc25898-fac3-4ac8-abac-fed00d81bbc0 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:41.551865 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc25898-fac3-4ac8-abac-fed00d81bbc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:41.579586 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-8dc25898-fac3-4ac8-abac-fed00d81bbc0 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:09:41.580966 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-8dc25898-fac3-4ac8-abac-fed00d81bbc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:09:41.584996 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b5f5eb-cb4d-4b84-b030-8214b7f6e9e8 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:41.589903 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 945/1910] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:09:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:09:41.591258 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc25898-fac3-4ac8-abac-fed00d81bbc0 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:41.597366 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333a42c3-4495-4a12-a320-e4224c917cdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:41.597675 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-8dc25898-fac3-4ac8-abac-fed00d81bbc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:09:41.598693 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333a42c3-4495-4a12-a320-e4224c917cdb admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:41.598982 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333a42c3-4495-4a12-a320-e4224c917cdb admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:41.599359 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333a42c3-4495-4a12-a320-e4224c917cdb admin admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:41.600256 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333a42c3-4495-4a12-a320-e4224c917cdb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:41.602371 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333a42c3-4495-4a12-a320-e4224c917cdb admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:41.618092 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc25898-fac3-4ac8-abac-fed00d81bbc0 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:41.629691 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac0147b-97d6-46f1-aba6-c17e7fade260 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:41.630436 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac0147b-97d6-46f1-aba6-c17e7fade260 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:41.630802 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac0147b-97d6-46f1-aba6-c17e7fade260 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:41.633487 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 966/1911] 10.222.0.46 () {60 vars in 1143 bytes} [Thu May 26 15:09:41 2022] POST /identity/v3/users => generated 297 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 15:09:41.633626 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac0147b-97d6-46f1-aba6-c17e7fade260 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:41.633626 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac0147b-97d6-46f1-aba6-c17e7fade260 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:41.633805 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc6af08-f685-4a8f-a3e8-f87e4d729fc6 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:41.633805 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc6af08-f685-4a8f-a3e8-f87e4d729fc6 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:41.633805 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc6af08-f685-4a8f-a3e8-f87e4d729fc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:41.633984 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac0147b-97d6-46f1-aba6-c17e7fade260 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:41.648765 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 946/1912] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 15:09:41 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 15:09:41.648765 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac0147b-97d6-46f1-aba6-c17e7fade260 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:41.653185 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-fbc6af08-f685-4a8f-a3e8-f87e4d729fc6 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:09:41.654439 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-fbc6af08-f685-4a8f-a3e8-f87e4d729fc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:09:41.658230 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac0147b-97d6-46f1-aba6-c17e7fade260 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:41.664103 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8de665-7a71-4bd2-9b04-1807873f5893 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:41.664532 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 967/1913] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:09:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:09:41.664782 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8de665-7a71-4bd2-9b04-1807873f5893 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:41.665073 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8de665-7a71-4bd2-9b04-1807873f5893 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:41.665356 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8de665-7a71-4bd2-9b04-1807873f5893 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:41.665557 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc6af08-f685-4a8f-a3e8-f87e4d729fc6 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:41.666233 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8de665-7a71-4bd2-9b04-1807873f5893 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:41.667855 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8de665-7a71-4bd2-9b04-1807873f5893 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:41.680423 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc6af08-f685-4a8f-a3e8-f87e4d729fc6 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:41.685018 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 947/1914] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 15:09:41 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 15:09:41.685018 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8de665-7a71-4bd2-9b04-1807873f5893 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:41.689320 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96602898-6752-4562-80f3-5a8332bd1d4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:41.690514 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96602898-6752-4562-80f3-5a8332bd1d4e admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:41.690801 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96602898-6752-4562-80f3-5a8332bd1d4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:41.691796 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96602898-6752-4562-80f3-5a8332bd1d4e admin admin] PATH_INFO: `/v3/projects/1927b6f4b6d44a6088c33e8c0d30cd3b/users/a9c2284769e842799979dfb913996f6b/roles/d4088d1845a24fbd9bd8975ce0b1d3d0` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:41.695268 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8de665-7a71-4bd2-9b04-1807873f5893 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:41.703122 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796e5233-2c7b-4dc5-acec-c2781a2c62e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:41.703803 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796e5233-2c7b-4dc5-acec-c2781a2c62e2 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:41.704082 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796e5233-2c7b-4dc5-acec-c2781a2c62e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:41.704356 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796e5233-2c7b-4dc5-acec-c2781a2c62e2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:41.705080 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796e5233-2c7b-4dc5-acec-c2781a2c62e2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 15:09:41.705385 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796e5233-2c7b-4dc5-acec-c2781a2c62e2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:41.707171 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796e5233-2c7b-4dc5-acec-c2781a2c62e2 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:41.712566 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96602898-6752-4562-80f3-5a8332bd1d4e admin admin] RBAC: Authorizing `identity:create_grant(project_id=1927b6f4b6d44a6088c33e8c0d30cd3b, user_id=a9c2284769e842799979dfb913996f6b, role_id=d4088d1845a24fbd9bd8975ce0b1d3d0)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:41.715858 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96602898-6752-4562-80f3-5a8332bd1d4e admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:41.718297 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:09:41.721095 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 948/1915] 10.222.0.46 () {58 vars in 1152 bytes} [Thu May 26 15:09:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 15:09:41.721583 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796e5233-2c7b-4dc5-acec-c2781a2c62e2 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:41.733088 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796e5233-2c7b-4dc5-acec-c2781a2c62e2 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:41.741845 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 968/1916] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 15:09:41 2022] PUT /identity/v3/projects/1927b6f4b6d44a6088c33e8c0d30cd3b/users/a9c2284769e842799979dfb913996f6b/roles/d4088d1845a24fbd9bd8975ce0b1d3d0 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:09:41.752369 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96602898-6752-4562-80f3-5a8332bd1d4e admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:41.817887 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbd38a5-b6aa-4a30-adc2-a9ebb88685bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:41.819658 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbd38a5-b6aa-4a30-adc2-a9ebb88685bb admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:41.820104 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbd38a5-b6aa-4a30-adc2-a9ebb88685bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:41.821140 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbd38a5-b6aa-4a30-adc2-a9ebb88685bb admin admin] PATH_INFO: `/v3/projects/658b0d6c1e9b46fe981657f449ef778e/users/bbb01d7afc0e4ce4a17a777bf6bea3ad/roles/a2e551773cbd4df084020676865b8222` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:41.838188 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96602898-6752-4562-80f3-5a8332bd1d4e admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:41.843670 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdbd38a5-b6aa-4a30-adc2-a9ebb88685bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=658b0d6c1e9b46fe981657f449ef778e, user_id=bbb01d7afc0e4ce4a17a777bf6bea3ad, role_id=a2e551773cbd4df084020676865b8222)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:41.846714 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdbd38a5-b6aa-4a30-adc2-a9ebb88685bb admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:41.847014 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bdd08f-1a94-4872-89d3-3e9b00ea3251 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:41.847856 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bdd08f-1a94-4872-89d3-3e9b00ea3251 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:41.848227 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bdd08f-1a94-4872-89d3-3e9b00ea3251 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:41.848754 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bdd08f-1a94-4872-89d3-3e9b00ea3251 admin admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:41.849665 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3bdd08f-1a94-4872-89d3-3e9b00ea3251 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:41.852106 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3bdd08f-1a94-4872-89d3-3e9b00ea3251 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:41.862093 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 949/1917] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 15:09:41 2022] PUT /identity/v3/projects/658b0d6c1e9b46fe981657f449ef778e/users/bbb01d7afc0e4ce4a17a777bf6bea3ad/roles/a2e551773cbd4df084020676865b8222 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:09:41.863405 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53d99ad-a3e7-4b09-93da-cfcfc1fb83fe None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:41.863729 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53d99ad-a3e7-4b09-93da-cfcfc1fb83fe None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:41.864008 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d53d99ad-a3e7-4b09-93da-cfcfc1fb83fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:41.877029 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:09:41.883854 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 969/1918] 10.222.0.46 () {60 vars in 1152 bytes} [Thu May 26 15:09:41 2022] POST /identity/v3/projects => generated 373 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 15:09:41.886818 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bdd08f-1a94-4872-89d3-3e9b00ea3251 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:41.900646 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-d53d99ad-a3e7-4b09-93da-cfcfc1fb83fe None None] MFA Rules not processed for user `a9c2284769e842799979dfb913996f6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:09:41.903704 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-d53d99ad-a3e7-4b09-93da-cfcfc1fb83fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:09:41.968586 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bdd08f-1a94-4872-89d3-3e9b00ea3251 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:41.980226 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c16bed-5810-4974-9f66-95dd439a8b51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:41.981716 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c16bed-5810-4974-9f66-95dd439a8b51 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:41.982154 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c16bed-5810-4974-9f66-95dd439a8b51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:41.982604 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c16bed-5810-4974-9f66-95dd439a8b51 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:41.984119 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c16bed-5810-4974-9f66-95dd439a8b51 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:41.985261 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c16bed-5810-4974-9f66-95dd439a8b51 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:41.995891 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 970/1919] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 15:09:41 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 15:09:41.997230 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c16bed-5810-4974-9f66-95dd439a8b51 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:42.008427 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c16bed-5810-4974-9f66-95dd439a8b51 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:42.009088 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 950/1920] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:09:41 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:09:42.010511 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53d99ad-a3e7-4b09-93da-cfcfc1fb83fe None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:42.021093 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e530505-7710-4093-80f9-ccd673de8765 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:42.022453 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e530505-7710-4093-80f9-ccd673de8765 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:42.023022 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e530505-7710-4093-80f9-ccd673de8765 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:42.023536 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e530505-7710-4093-80f9-ccd673de8765 admin admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:42.024566 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e530505-7710-4093-80f9-ccd673de8765 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:42.027572 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e530505-7710-4093-80f9-ccd673de8765 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:42.035529 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d53d99ad-a3e7-4b09-93da-cfcfc1fb83fe None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:42.045870 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0081b58-1315-4c7f-a5a0-791867a5dfea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:42.046813 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0081b58-1315-4c7f-a5a0-791867a5dfea admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:42.047114 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0081b58-1315-4c7f-a5a0-791867a5dfea admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:42.047696 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0081b58-1315-4c7f-a5a0-791867a5dfea admin admin] PATH_INFO: `/v3/projects/658b0d6c1e9b46fe981657f449ef778e/users/bbb01d7afc0e4ce4a17a777bf6bea3ad/roles/a2e551773cbd4df084020676865b8222` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:42.050120 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0081b58-1315-4c7f-a5a0-791867a5dfea admin admin] RBAC: Authorizing `identity:create_grant(project_id=658b0d6c1e9b46fe981657f449ef778e, user_id=bbb01d7afc0e4ce4a17a777bf6bea3ad, role_id=a2e551773cbd4df084020676865b8222)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:42.053085 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0081b58-1315-4c7f-a5a0-791867a5dfea admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:42.062838 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 971/1921] 10.222.0.46 () {60 vars in 1143 bytes} [Thu May 26 15:09:41 2022] POST /identity/v3/users => generated 287 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 15:09:42.064019 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e530505-7710-4093-80f9-ccd673de8765 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:42.077965 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 951/1922] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 15:09:42 2022] PUT /identity/v3/projects/658b0d6c1e9b46fe981657f449ef778e/users/bbb01d7afc0e4ce4a17a777bf6bea3ad/roles/a2e551773cbd4df084020676865b8222 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:09:42.077965 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0081b58-1315-4c7f-a5a0-791867a5dfea admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:42.078951 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e530505-7710-4093-80f9-ccd673de8765 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:42.167009 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:INSERT=5,keystone:SELECT=95 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:09:42.174309 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0081b58-1315-4c7f-a5a0-791867a5dfea admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:42.178316 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efa76d3-6102-4761-8edf-0eb9ec7b36ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:42.179895 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efa76d3-6102-4761-8edf-0eb9ec7b36ef admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:42.180266 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efa76d3-6102-4761-8edf-0eb9ec7b36ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:42.181269 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efa76d3-6102-4761-8edf-0eb9ec7b36ef admin admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:42.187632 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efa76d3-6102-4761-8edf-0eb9ec7b36ef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:42.187632 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efa76d3-6102-4761-8edf-0eb9ec7b36ef admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:42.195328 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba2ad9f-df01-4e05-9ffb-3a7fa843925e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:42.196262 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba2ad9f-df01-4e05-9ffb-3a7fa843925e admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:42.196588 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba2ad9f-df01-4e05-9ffb-3a7fa843925e admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:42.196895 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba2ad9f-df01-4e05-9ffb-3a7fa843925e admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:42.197857 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba2ad9f-df01-4e05-9ffb-3a7fa843925e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:42.199815 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba2ad9f-df01-4e05-9ffb-3a7fa843925e admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:42.216060 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 952/1923] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 15:09:42 2022] GET /identity/v3/roles => generated 1420 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 15:09:42.217549 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7acfad8-5b9f-4889-aeba-c0b02b00dab1 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:42.217847 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7acfad8-5b9f-4889-aeba-c0b02b00dab1 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:42.218144 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7acfad8-5b9f-4889-aeba-c0b02b00dab1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:42.252913 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 972/1924] 10.222.0.46 () {60 vars in 1152 bytes} [Thu May 26 15:09:42 2022] POST /identity/v3/projects => generated 381 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 15:09:42.259930 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efa76d3-6102-4761-8edf-0eb9ec7b36ef admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:42.279533 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-e7acfad8-5b9f-4889-aeba-c0b02b00dab1 None None] MFA Rules not processed for user `bbb01d7afc0e4ce4a17a777bf6bea3ad`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:09:42.280991 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-e7acfad8-5b9f-4889-aeba-c0b02b00dab1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:09:42.351182 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efa76d3-6102-4761-8edf-0eb9ec7b36ef admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:42.361208 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de09707-49eb-4b5a-a47d-1bf12c88aa56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:42.361932 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de09707-49eb-4b5a-a47d-1bf12c88aa56 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:42.362220 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de09707-49eb-4b5a-a47d-1bf12c88aa56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:42.362498 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de09707-49eb-4b5a-a47d-1bf12c88aa56 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:42.363237 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de09707-49eb-4b5a-a47d-1bf12c88aa56 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:42.364858 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de09707-49eb-4b5a-a47d-1bf12c88aa56 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:42.380789 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 973/1925] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 15:09:42 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 15:09:42.380789 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de09707-49eb-4b5a-a47d-1bf12c88aa56 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:42.386415 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 953/1926] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:09:42 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:09:42.387991 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7acfad8-5b9f-4889-aeba-c0b02b00dab1 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:42.389615 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-e7acfad8-5b9f-4889-aeba-c0b02b00dab1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:09:42.397199 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:09:42.401787 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de09707-49eb-4b5a-a47d-1bf12c88aa56 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:42.409509 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58776040-ca96-49c1-8e63-59b221fd1ca6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:42.410516 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58776040-ca96-49c1-8e63-59b221fd1ca6 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:42.410908 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58776040-ca96-49c1-8e63-59b221fd1ca6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:42.411198 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58776040-ca96-49c1-8e63-59b221fd1ca6 admin admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:42.412418 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58776040-ca96-49c1-8e63-59b221fd1ca6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:42.414633 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58776040-ca96-49c1-8e63-59b221fd1ca6 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:42.455691 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 974/1927] 10.222.0.46 () {60 vars in 1143 bytes} [Thu May 26 15:09:42 2022] POST /identity/v3/users => generated 297 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 15:09:42.457218 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58776040-ca96-49c1-8e63-59b221fd1ca6 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:42.472305 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58776040-ca96-49c1-8e63-59b221fd1ca6 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:42.478658 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96356597-08ab-4eb0-8a97-8b29201a4eaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:42.481492 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96356597-08ab-4eb0-8a97-8b29201a4eaf admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:42.481941 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96356597-08ab-4eb0-8a97-8b29201a4eaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:42.482349 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96356597-08ab-4eb0-8a97-8b29201a4eaf admin admin] PATH_INFO: `/v3/projects/71a469d15aa642e9b2aa1ec3bbf47ec8/users/9de6d63de3eb48f8a887c57b33d3a78e/roles/d4088d1845a24fbd9bd8975ce0b1d3d0` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:42.487677 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7acfad8-5b9f-4889-aeba-c0b02b00dab1 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:42.494511 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933c6cda-da39-49ca-99c4-137a14db242b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5b5b35614a547d783e6e77ddb8c1c03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:42.495499 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933c6cda-da39-49ca-99c4-137a14db242b service cinder] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:42.495793 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933c6cda-da39-49ca-99c4-137a14db242b service cinder] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:42.496194 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933c6cda-da39-49ca-99c4-137a14db242b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:42.499057 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-933c6cda-da39-49ca-99c4-137a14db242b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:09:42.510885 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96356597-08ab-4eb0-8a97-8b29201a4eaf admin admin] RBAC: Authorizing `identity:create_grant(project_id=71a469d15aa642e9b2aa1ec3bbf47ec8, user_id=9de6d63de3eb48f8a887c57b33d3a78e, role_id=d4088d1845a24fbd9bd8975ce0b1d3d0)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:42.514825 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96356597-08ab-4eb0-8a97-8b29201a4eaf admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:42.532919 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 975/1928] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 15:09:42 2022] PUT /identity/v3/projects/71a469d15aa642e9b2aa1ec3bbf47ec8/users/9de6d63de3eb48f8a887c57b33d3a78e/roles/d4088d1845a24fbd9bd8975ce0b1d3d0 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:09:42.534231 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96356597-08ab-4eb0-8a97-8b29201a4eaf admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:42.536542 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-96356597-08ab-4eb0-8a97-8b29201a4eaf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:09:42.623062 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933c6cda-da39-49ca-99c4-137a14db242b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:42.625872 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933c6cda-da39-49ca-99c4-137a14db242b service cinder] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:42.639945 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 954/1929] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:09:42 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 253 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 15:09:42.640945 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933c6cda-da39-49ca-99c4-137a14db242b service cinder] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:42.666849 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933c6cda-da39-49ca-99c4-137a14db242b service cinder] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:42.671064 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96356597-08ab-4eb0-8a97-8b29201a4eaf admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:42.678401 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5389ed7a-f544-434b-aa1c-3f41b8f78aaa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5b5b35614a547d783e6e77ddb8c1c03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:42.679076 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5389ed7a-f544-434b-aa1c-3f41b8f78aaa service cinder] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:42.679358 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5389ed7a-f544-434b-aa1c-3f41b8f78aaa service cinder] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:42.679635 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5389ed7a-f544-434b-aa1c-3f41b8f78aaa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:42.781964 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:09:42.811714 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5389ed7a-f544-434b-aa1c-3f41b8f78aaa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:42.815400 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5389ed7a-f544-434b-aa1c-3f41b8f78aaa service cinder] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:42.832263 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae2e8f1-2066-43e1-9bda-06586aa647b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:42.833275 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae2e8f1-2066-43e1-9bda-06586aa647b9 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:42.833688 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae2e8f1-2066-43e1-9bda-06586aa647b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:42.834184 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae2e8f1-2066-43e1-9bda-06586aa647b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:42.835081 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae2e8f1-2066-43e1-9bda-06586aa647b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:42.837347 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae2e8f1-2066-43e1-9bda-06586aa647b9 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:42.850541 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 976/1930] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:09:42 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 15:09:42.852919 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4874b523-99e9-4bcd-a5de-ad42d21a2b01 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:42.852919 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4874b523-99e9-4bcd-a5de-ad42d21a2b01 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:42.852919 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4874b523-99e9-4bcd-a5de-ad42d21a2b01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:42.867868 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 955/1931] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 15:09:42 2022] GET /identity/v3/roles => generated 1420 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 15:09:42.872844 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae2e8f1-2066-43e1-9bda-06586aa647b9 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:42.906225 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-4874b523-99e9-4bcd-a5de-ad42d21a2b01 None None] MFA Rules not processed for user `9de6d63de3eb48f8a887c57b33d3a78e`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:09:42.907636 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-4874b523-99e9-4bcd-a5de-ad42d21a2b01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:09:42.914952 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae2e8f1-2066-43e1-9bda-06586aa647b9 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:42.932523 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5838f87-e087-4f56-a758-c6a749120d87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:42.935308 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5838f87-e087-4f56-a758-c6a749120d87 admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:42.935673 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5838f87-e087-4f56-a758-c6a749120d87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:42.935922 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5838f87-e087-4f56-a758-c6a749120d87 admin admin] PATH_INFO: `/v3/projects/d0ab6453132948038ccb7f4e8176e5e7/users/818797c803554fe292ad3b8ab6f90fbf/roles/a2e551773cbd4df084020676865b8222` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:42.955106 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5838f87-e087-4f56-a758-c6a749120d87 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d0ab6453132948038ccb7f4e8176e5e7, user_id=818797c803554fe292ad3b8ab6f90fbf, role_id=a2e551773cbd4df084020676865b8222)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:42.959488 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5838f87-e087-4f56-a758-c6a749120d87 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:43.016093 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 956/1932] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 15:09:42 2022] PUT /identity/v3/projects/d0ab6453132948038ccb7f4e8176e5e7/users/818797c803554fe292ad3b8ab6f90fbf/roles/a2e551773cbd4df084020676865b8222 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:09:43.020352 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5838f87-e087-4f56-a758-c6a749120d87 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:43.061844 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 977/1933] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:09:42 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:09:43.062733 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4874b523-99e9-4bcd-a5de-ad42d21a2b01 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:43.076011 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4874b523-99e9-4bcd-a5de-ad42d21a2b01 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:43.088756 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:09:43.135965 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5838f87-e087-4f56-a758-c6a749120d87 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:43.143682 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077c6a55-d42c-4300-86b9-2a60c57e4c02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:43.144497 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077c6a55-d42c-4300-86b9-2a60c57e4c02 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:43.145972 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077c6a55-d42c-4300-86b9-2a60c57e4c02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:43.145972 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077c6a55-d42c-4300-86b9-2a60c57e4c02 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:43.146268 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077c6a55-d42c-4300-86b9-2a60c57e4c02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:43.147947 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077c6a55-d42c-4300-86b9-2a60c57e4c02 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:43.164635 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6fd129-b617-4970-9570-47bea6229f85 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5b5b35614a547d783e6e77ddb8c1c03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:43.169681 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6fd129-b617-4970-9570-47bea6229f85 service cinder] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:43.169681 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6fd129-b617-4970-9570-47bea6229f85 service cinder] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:43.169681 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6fd129-b617-4970-9570-47bea6229f85 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:43.170245 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 957/1934] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 15:09:43 2022] GET /identity/v3/roles => generated 1420 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 15:09:43.171078 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077c6a55-d42c-4300-86b9-2a60c57e4c02 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:43.174608 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-2a6fd129-b617-4970-9570-47bea6229f85 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:09:43.185895 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077c6a55-d42c-4300-86b9-2a60c57e4c02 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:43.191032 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1e1c6f-1a88-415b-af71-b436d93bd573 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:43.191938 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1e1c6f-1a88-415b-af71-b436d93bd573 admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:43.192225 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1e1c6f-1a88-415b-af71-b436d93bd573 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:43.192503 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1e1c6f-1a88-415b-af71-b436d93bd573 admin admin] PATH_INFO: `/v3/projects/d0ab6453132948038ccb7f4e8176e5e7/users/818797c803554fe292ad3b8ab6f90fbf/roles/a2e551773cbd4df084020676865b8222` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:43.194713 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1e1c6f-1a88-415b-af71-b436d93bd573 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d0ab6453132948038ccb7f4e8176e5e7, user_id=818797c803554fe292ad3b8ab6f90fbf, role_id=a2e551773cbd4df084020676865b8222)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:43.198066 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1e1c6f-1a88-415b-af71-b436d93bd573 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:43.216804 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 958/1935] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 15:09:43 2022] PUT /identity/v3/projects/d0ab6453132948038ccb7f4e8176e5e7/users/818797c803554fe292ad3b8ab6f90fbf/roles/a2e551773cbd4df084020676865b8222 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:09:43.227391 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7979f7c3-0664-47ed-9adf-37ee1fe5a4d2 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:43.227697 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7979f7c3-0664-47ed-9adf-37ee1fe5a4d2 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:43.227945 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7979f7c3-0664-47ed-9adf-37ee1fe5a4d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:43.278575 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6fd129-b617-4970-9570-47bea6229f85 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:43.278817 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-7979f7c3-0664-47ed-9adf-37ee1fe5a4d2 None None] MFA Rules not processed for user `818797c803554fe292ad3b8ab6f90fbf`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:09:43.280119 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6fd129-b617-4970-9570-47bea6229f85 service cinder] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:43.280225 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-7979f7c3-0664-47ed-9adf-37ee1fe5a4d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:09:43.297519 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 978/1936] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:09:43 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 15:09:43.421485 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 959/1937] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:09:43 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:09:43.488621 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6fd129-b617-4970-9570-47bea6229f85 service cinder] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:43.499336 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6fd129-b617-4970-9570-47bea6229f85 service cinder] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:43.541184 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:09:43.568218 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc16b213-abcb-4e9b-acbe-33e7e93aa0cd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5b5b35614a547d783e6e77ddb8c1c03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:43.569526 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc16b213-abcb-4e9b-acbe-33e7e93aa0cd service cinder] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:43.569822 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc16b213-abcb-4e9b-acbe-33e7e93aa0cd service cinder] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:43.570105 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc16b213-abcb-4e9b-acbe-33e7e93aa0cd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:43.679202 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc16b213-abcb-4e9b-acbe-33e7e93aa0cd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:43.682668 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc16b213-abcb-4e9b-acbe-33e7e93aa0cd service cinder] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:43.719358 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 979/1938] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:09:43 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 231 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 15:09:43.747604 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7979f7c3-0664-47ed-9adf-37ee1fe5a4d2 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:43.749598 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc16b213-abcb-4e9b-acbe-33e7e93aa0cd service cinder] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:43.773226 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7979f7c3-0664-47ed-9adf-37ee1fe5a4d2 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:43.786604 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9553d4d6-7355-4347-82b3-74cd0f77b1ad service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5b5b35614a547d783e6e77ddb8c1c03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:43.787203 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9553d4d6-7355-4347-82b3-74cd0f77b1ad service cinder] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:43.787508 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9553d4d6-7355-4347-82b3-74cd0f77b1ad service cinder] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:43.787817 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9553d4d6-7355-4347-82b3-74cd0f77b1ad service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:43.813326 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9553d4d6-7355-4347-82b3-74cd0f77b1ad service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:43.818560 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9553d4d6-7355-4347-82b3-74cd0f77b1ad service cinder] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:43.870003 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 960/1939] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:09:43 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 15:09:43.919973 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc16b213-abcb-4e9b-acbe-33e7e93aa0cd service cinder] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:43.938071 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef6a707-82e3-41e7-a138-169f6fa7a080 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:43.938974 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef6a707-82e3-41e7-a138-169f6fa7a080 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:43.939262 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef6a707-82e3-41e7-a138-169f6fa7a080 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:43.939548 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef6a707-82e3-41e7-a138-169f6fa7a080 admin admin] PATH_INFO: `/v3/users/9de6d63de3eb48f8a887c57b33d3a78e` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:43.948066 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef6a707-82e3-41e7-a138-169f6fa7a080 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9de6d63de3eb48f8a887c57b33d3a78e)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:43.948066 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef6a707-82e3-41e7-a138-169f6fa7a080 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:44.046686 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-eef6a707-82e3-41e7-a138-169f6fa7a080 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9de6d63de3eb48f8a887c57b33d3a78e'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:44.060421 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-eef6a707-82e3-41e7-a138-169f6fa7a080 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9de6d63de3eb48f8a887c57b33d3a78e'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:44.061703 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-eef6a707-82e3-41e7-a138-169f6fa7a080 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9de6d63de3eb48f8a887c57b33d3a78e'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:44.084725 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-eef6a707-82e3-41e7-a138-169f6fa7a080 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9de6d63de3eb48f8a887c57b33d3a78e'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:44.094014 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9553d4d6-7355-4347-82b3-74cd0f77b1ad service cinder] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:44.111724 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9553d4d6-7355-4347-82b3-74cd0f77b1ad service cinder] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:44.117316 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-eef6a707-82e3-41e7-a138-169f6fa7a080 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9de6d63de3eb48f8a887c57b33d3a78e'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:44.122111 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140037ec-90d2-4d0d-b381-15901b53a164 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5b5b35614a547d783e6e77ddb8c1c03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:44.123671 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140037ec-90d2-4d0d-b381-15901b53a164 service cinder] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:44.124739 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140037ec-90d2-4d0d-b381-15901b53a164 service cinder] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:44.125336 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140037ec-90d2-4d0d-b381-15901b53a164 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:44.130870 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-140037ec-90d2-4d0d-b381-15901b53a164 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:09:44.136312 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 980/1940] 10.222.0.46 () {58 vars in 1223 bytes} [Thu May 26 15:09:43 2022] DELETE /identity/v3/users/9de6d63de3eb48f8a887c57b33d3a78e => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:09:44.138932 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cb2eeb-ef9f-47a2-b036-d676db746769 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:44.139374 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cb2eeb-ef9f-47a2-b036-d676db746769 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:44.139908 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cb2eeb-ef9f-47a2-b036-d676db746769 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:44.147497 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:09:44.171802 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-15cb2eeb-ef9f-47a2-b036-d676db746769 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:09:44.173559 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-15cb2eeb-ef9f-47a2-b036-d676db746769 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:09:44.266769 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140037ec-90d2-4d0d-b381-15901b53a164 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:44.270651 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140037ec-90d2-4d0d-b381-15901b53a164 service cinder] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:44.301100 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 961/1941] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:09:44 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 15:09:44.325146 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 981/1942] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:09:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:09:44.403785 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140037ec-90d2-4d0d-b381-15901b53a164 service cinder] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:44.405748 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-140037ec-90d2-4d0d-b381-15901b53a164 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:09:44.436141 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:09:44.513932 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140037ec-90d2-4d0d-b381-15901b53a164 service cinder] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:44.564390 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cb2eeb-ef9f-47a2-b036-d676db746769 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:44.566462 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-15cb2eeb-ef9f-47a2-b036-d676db746769 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:09:44.593776 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3784ab-0cdf-4bd6-afed-ca675dca6148 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0156a2f6f3794f479e76b85bdbbe5c26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:44.594486 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3784ab-0cdf-4bd6-afed-ca675dca6148 service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:44.594865 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3784ab-0cdf-4bd6-afed-ca675dca6148 service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:44.595228 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3784ab-0cdf-4bd6-afed-ca675dca6148 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:44.613885 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3784ab-0cdf-4bd6-afed-ca675dca6148 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:44.615358 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3784ab-0cdf-4bd6-afed-ca675dca6148 service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:44.632362 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 962/1943] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:09:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 15:09:44.648224 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cb2eeb-ef9f-47a2-b036-d676db746769 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:44.653952 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcdafa1-4761-4b4f-a1c1-f99ba19b8a73 tempest-VolumeQuotaClassesTest-2130536466 tempest-VolumeQuotaClassesTest-2130536466-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818797c803554fe292ad3b8ab6f90fbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0ab6453132948038ccb7f4e8176e5e7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:44.654662 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcdafa1-4761-4b4f-a1c1-f99ba19b8a73 tempest-VolumeQuotaClassesTest-2130536466 tempest-VolumeQuotaClassesTest-2130536466-project-admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:44.654945 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcdafa1-4761-4b4f-a1c1-f99ba19b8a73 tempest-VolumeQuotaClassesTest-2130536466 tempest-VolumeQuotaClassesTest-2130536466-project-admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:44.655309 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcdafa1-4761-4b4f-a1c1-f99ba19b8a73 tempest-VolumeQuotaClassesTest-2130536466 tempest-VolumeQuotaClassesTest-2130536466-project-admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:44.656002 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffcdafa1-4761-4b4f-a1c1-f99ba19b8a73 tempest-VolumeQuotaClassesTest-2130536466 tempest-VolumeQuotaClassesTest-2130536466-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 15:09:44.656310 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffcdafa1-4761-4b4f-a1c1-f99ba19b8a73 tempest-VolumeQuotaClassesTest-2130536466 tempest-VolumeQuotaClassesTest-2130536466-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:44.658416 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffcdafa1-4761-4b4f-a1c1-f99ba19b8a73 tempest-VolumeQuotaClassesTest-2130536466 tempest-VolumeQuotaClassesTest-2130536466-project-admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:44.671998 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 982/1944] 10.222.0.46 () {58 vars in 1152 bytes} [Thu May 26 15:09:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 15:09:44.678146 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3784ab-0cdf-4bd6-afed-ca675dca6148 service neutron] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:44.690923 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3784ab-0cdf-4bd6-afed-ca675dca6148 service neutron] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:44.699058 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24baadeb-343e-4150-aef6-1589f07b9376 tempest-VolumeQuotaClassesTest-2130536466 tempest-VolumeQuotaClassesTest-2130536466-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818797c803554fe292ad3b8ab6f90fbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0ab6453132948038ccb7f4e8176e5e7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:44.700419 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24baadeb-343e-4150-aef6-1589f07b9376 tempest-VolumeQuotaClassesTest-2130536466 tempest-VolumeQuotaClassesTest-2130536466-project-admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:44.700854 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24baadeb-343e-4150-aef6-1589f07b9376 tempest-VolumeQuotaClassesTest-2130536466 tempest-VolumeQuotaClassesTest-2130536466-project-admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:44.701334 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24baadeb-343e-4150-aef6-1589f07b9376 tempest-VolumeQuotaClassesTest-2130536466 tempest-VolumeQuotaClassesTest-2130536466-project-admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:44.702249 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24baadeb-343e-4150-aef6-1589f07b9376 tempest-VolumeQuotaClassesTest-2130536466 tempest-VolumeQuotaClassesTest-2130536466-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:44.704702 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24baadeb-343e-4150-aef6-1589f07b9376 tempest-VolumeQuotaClassesTest-2130536466 tempest-VolumeQuotaClassesTest-2130536466-project-admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:44.742631 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 963/1945] 10.222.0.46 () {60 vars in 1152 bytes} [Thu May 26 15:09:44 2022] POST /identity/v3/projects => generated 355 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 15:09:44.744088 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcdafa1-4761-4b4f-a1c1-f99ba19b8a73 tempest-VolumeQuotaClassesTest-2130536466 tempest-VolumeQuotaClassesTest-2130536466-project-admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:44.758289 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcdafa1-4761-4b4f-a1c1-f99ba19b8a73 tempest-VolumeQuotaClassesTest-2130536466 tempest-VolumeQuotaClassesTest-2130536466-project-admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:44.853872 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f030f7c-4991-4a7a-8199-7719c8901f32 tempest-VolumeQuotaClassesTest-2130536466 tempest-VolumeQuotaClassesTest-2130536466-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818797c803554fe292ad3b8ab6f90fbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0ab6453132948038ccb7f4e8176e5e7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:44.854485 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f030f7c-4991-4a7a-8199-7719c8901f32 tempest-VolumeQuotaClassesTest-2130536466 tempest-VolumeQuotaClassesTest-2130536466-project-admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:44.854775 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f030f7c-4991-4a7a-8199-7719c8901f32 tempest-VolumeQuotaClassesTest-2130536466 tempest-VolumeQuotaClassesTest-2130536466-project-admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:44.855104 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f030f7c-4991-4a7a-8199-7719c8901f32 tempest-VolumeQuotaClassesTest-2130536466 tempest-VolumeQuotaClassesTest-2130536466-project-admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:44.856609 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f030f7c-4991-4a7a-8199-7719c8901f32 tempest-VolumeQuotaClassesTest-2130536466 tempest-VolumeQuotaClassesTest-2130536466-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 15:09:44.856701 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f030f7c-4991-4a7a-8199-7719c8901f32 tempest-VolumeQuotaClassesTest-2130536466 tempest-VolumeQuotaClassesTest-2130536466-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:44.858671 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f030f7c-4991-4a7a-8199-7719c8901f32 tempest-VolumeQuotaClassesTest-2130536466 tempest-VolumeQuotaClassesTest-2130536466-project-admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:44.875977 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 983/1946] 10.222.0.46 () {58 vars in 1152 bytes} [Thu May 26 15:09:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 15:09:44.963618 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24baadeb-343e-4150-aef6-1589f07b9376 tempest-VolumeQuotaClassesTest-2130536466 tempest-VolumeQuotaClassesTest-2130536466-project-admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:44.977644 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24baadeb-343e-4150-aef6-1589f07b9376 tempest-VolumeQuotaClassesTest-2130536466 tempest-VolumeQuotaClassesTest-2130536466-project-admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:44.987538 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c30044c-76aa-42b2-8f2d-1e2aa5fb39f5 tempest-VolumeQuotaClassesTest-2130536466 tempest-VolumeQuotaClassesTest-2130536466-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818797c803554fe292ad3b8ab6f90fbf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'd0ab6453132948038ccb7f4e8176e5e7', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:44.988708 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c30044c-76aa-42b2-8f2d-1e2aa5fb39f5 tempest-VolumeQuotaClassesTest-2130536466 tempest-VolumeQuotaClassesTest-2130536466-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:44.989132 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c30044c-76aa-42b2-8f2d-1e2aa5fb39f5 tempest-VolumeQuotaClassesTest-2130536466 tempest-VolumeQuotaClassesTest-2130536466-project-admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:44.989422 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c30044c-76aa-42b2-8f2d-1e2aa5fb39f5 tempest-VolumeQuotaClassesTest-2130536466 tempest-VolumeQuotaClassesTest-2130536466-project-admin] PATH_INFO: `/v3/projects/0ba4ed7a7df04edbb3e520a97acfb784` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:44.991276 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c30044c-76aa-42b2-8f2d-1e2aa5fb39f5 tempest-VolumeQuotaClassesTest-2130536466 tempest-VolumeQuotaClassesTest-2130536466-project-admin] RBAC: Authorizing `identity:delete_project(project_id=0ba4ed7a7df04edbb3e520a97acfb784)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:44.994275 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c30044c-76aa-42b2-8f2d-1e2aa5fb39f5 tempest-VolumeQuotaClassesTest-2130536466 tempest-VolumeQuotaClassesTest-2130536466-project-admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:45.138485 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-9c30044c-76aa-42b2-8f2d-1e2aa5fb39f5 tempest-VolumeQuotaClassesTest-2130536466 tempest-VolumeQuotaClassesTest-2130536466-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0ba4ed7a7df04edbb3e520a97acfb784', 'request_id': 'req-9c30044c-76aa-42b2-8f2d-1e2aa5fb39f5'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:45.160788 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-9c30044c-76aa-42b2-8f2d-1e2aa5fb39f5 tempest-VolumeQuotaClassesTest-2130536466 tempest-VolumeQuotaClassesTest-2130536466-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0ba4ed7a7df04edbb3e520a97acfb784', 'request_id': 'req-9c30044c-76aa-42b2-8f2d-1e2aa5fb39f5'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:45.174185 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-9c30044c-76aa-42b2-8f2d-1e2aa5fb39f5 tempest-VolumeQuotaClassesTest-2130536466 tempest-VolumeQuotaClassesTest-2130536466-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0ba4ed7a7df04edbb3e520a97acfb784', 'request_id': 'req-9c30044c-76aa-42b2-8f2d-1e2aa5fb39f5'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:45.194039 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-9c30044c-76aa-42b2-8f2d-1e2aa5fb39f5 tempest-VolumeQuotaClassesTest-2130536466 tempest-VolumeQuotaClassesTest-2130536466-project-admin] The token cache is being invalidate because project 0ba4ed7a7df04edbb3e520a97acfb784 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 15:09:45.194438 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-9c30044c-76aa-42b2-8f2d-1e2aa5fb39f5 tempest-VolumeQuotaClassesTest-2130536466 tempest-VolumeQuotaClassesTest-2130536466-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:45.195692 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-9c30044c-76aa-42b2-8f2d-1e2aa5fb39f5 tempest-VolumeQuotaClassesTest-2130536466 tempest-VolumeQuotaClassesTest-2130536466-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:45.202811 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 964/1947] 10.222.0.46 () {58 vars in 1232 bytes} [Thu May 26 15:09:44 2022] DELETE /identity/v3/projects/0ba4ed7a7df04edbb3e520a97acfb784 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:09:45.241042 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f030f7c-4991-4a7a-8199-7719c8901f32 tempest-VolumeQuotaClassesTest-2130536466 tempest-VolumeQuotaClassesTest-2130536466-project-admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:45.243158 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-5f030f7c-4991-4a7a-8199-7719c8901f32 tempest-VolumeQuotaClassesTest-2130536466 tempest-VolumeQuotaClassesTest-2130536466-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:09:45.329233 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:09:45.398899 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f030f7c-4991-4a7a-8199-7719c8901f32 tempest-VolumeQuotaClassesTest-2130536466 tempest-VolumeQuotaClassesTest-2130536466-project-admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:45.609956 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ef456d-2c2c-4ae7-8124-59e60ca54d0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:45.611109 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ef456d-2c2c-4ae7-8124-59e60ca54d0c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:45.611490 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ef456d-2c2c-4ae7-8124-59e60ca54d0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:45.612311 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ef456d-2c2c-4ae7-8124-59e60ca54d0c admin admin] PATH_INFO: `/v3/projects/71a469d15aa642e9b2aa1ec3bbf47ec8` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:45.617650 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ef456d-2c2c-4ae7-8124-59e60ca54d0c admin admin] RBAC: Authorizing `identity:delete_project(project_id=71a469d15aa642e9b2aa1ec3bbf47ec8)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:45.623884 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ef456d-2c2c-4ae7-8124-59e60ca54d0c admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:45.757123 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c30044c-76aa-42b2-8f2d-1e2aa5fb39f5 tempest-VolumeQuotaClassesTest-2130536466 tempest-VolumeQuotaClassesTest-2130536466-project-admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:45.759987 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-9c30044c-76aa-42b2-8f2d-1e2aa5fb39f5 tempest-VolumeQuotaClassesTest-2130536466 tempest-VolumeQuotaClassesTest-2130536466-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:09:45.813023 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c30044c-76aa-42b2-8f2d-1e2aa5fb39f5 tempest-VolumeQuotaClassesTest-2130536466 tempest-VolumeQuotaClassesTest-2130536466-project-admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:45.813270 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-72ef456d-2c2c-4ae7-8124-59e60ca54d0c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '71a469d15aa642e9b2aa1ec3bbf47ec8', 'request_id': 'req-72ef456d-2c2c-4ae7-8124-59e60ca54d0c'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:45.835197 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-72ef456d-2c2c-4ae7-8124-59e60ca54d0c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '71a469d15aa642e9b2aa1ec3bbf47ec8', 'request_id': 'req-72ef456d-2c2c-4ae7-8124-59e60ca54d0c'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:45.848009 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-72ef456d-2c2c-4ae7-8124-59e60ca54d0c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '71a469d15aa642e9b2aa1ec3bbf47ec8', 'request_id': 'req-72ef456d-2c2c-4ae7-8124-59e60ca54d0c'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:45.913814 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-72ef456d-2c2c-4ae7-8124-59e60ca54d0c admin admin] The token cache is being invalidate because project 71a469d15aa642e9b2aa1ec3bbf47ec8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 15:09:45.913814 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-72ef456d-2c2c-4ae7-8124-59e60ca54d0c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:45.916157 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-72ef456d-2c2c-4ae7-8124-59e60ca54d0c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:45.923439 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 984/1948] 10.222.0.46 () {58 vars in 1232 bytes} [Thu May 26 15:09:45 2022] DELETE /identity/v3/projects/71a469d15aa642e9b2aa1ec3bbf47ec8 => generated 0 bytes in 679 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:09:45.925962 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de0570a-70bf-478d-ab8b-d573be9fc170 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:45.927166 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de0570a-70bf-478d-ab8b-d573be9fc170 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:45.927166 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de0570a-70bf-478d-ab8b-d573be9fc170 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:45.937263 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:09:45.988895 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8b2717-05ab-4575-a08b-04568b7cfc04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:45.992409 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8b2717-05ab-4575-a08b-04568b7cfc04 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:45.992907 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8b2717-05ab-4575-a08b-04568b7cfc04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:45.994174 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8b2717-05ab-4575-a08b-04568b7cfc04 admin admin] PATH_INFO: `/v3/users/a9c2284769e842799979dfb913996f6b` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:46.009489 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8b2717-05ab-4575-a08b-04568b7cfc04 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a9c2284769e842799979dfb913996f6b)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:46.009489 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8b2717-05ab-4575-a08b-04568b7cfc04 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:46.019220 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-0de0570a-70bf-478d-ab8b-d573be9fc170 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:09:46.023648 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-0de0570a-70bf-478d-ab8b-d573be9fc170 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:09:46.052496 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 985/1949] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:09:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:09:46.061703 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8bd8b3-04b3-4dd7-893c-f6c6663d3556 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:46.061988 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8bd8b3-04b3-4dd7-893c-f6c6663d3556 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:46.062326 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8bd8b3-04b3-4dd7-893c-f6c6663d3556 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:46.097323 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-ca8bd8b3-04b3-4dd7-893c-f6c6663d3556 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:09:46.099244 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-ca8bd8b3-04b3-4dd7-893c-f6c6663d3556 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:09:46.116693 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 986/1950] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:09:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:09:46.137614 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8bd8b3-04b3-4dd7-893c-f6c6663d3556 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:46.170396 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-9c8b2717-05ab-4575-a08b-04568b7cfc04 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a9c2284769e842799979dfb913996f6b'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:46.175101 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8bd8b3-04b3-4dd7-893c-f6c6663d3556 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:46.190118 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664dc428-b82e-4bdd-90aa-45ac2fc59030 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:46.190807 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664dc428-b82e-4bdd-90aa-45ac2fc59030 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:46.191099 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664dc428-b82e-4bdd-90aa-45ac2fc59030 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:46.191980 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664dc428-b82e-4bdd-90aa-45ac2fc59030 admin admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:46.192735 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664dc428-b82e-4bdd-90aa-45ac2fc59030 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 15:09:46.193048 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664dc428-b82e-4bdd-90aa-45ac2fc59030 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:46.196903 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664dc428-b82e-4bdd-90aa-45ac2fc59030 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:46.201721 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-9c8b2717-05ab-4575-a08b-04568b7cfc04 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a9c2284769e842799979dfb913996f6b'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:46.202840 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-9c8b2717-05ab-4575-a08b-04568b7cfc04 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a9c2284769e842799979dfb913996f6b'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:46.224167 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 987/1951] 10.222.0.46 () {58 vars in 1152 bytes} [Thu May 26 15:09:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 15:09:46.226732 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664dc428-b82e-4bdd-90aa-45ac2fc59030 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:46.231791 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-664dc428-b82e-4bdd-90aa-45ac2fc59030 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:09:46.240731 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-9c8b2717-05ab-4575-a08b-04568b7cfc04 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a9c2284769e842799979dfb913996f6b'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:46.259034 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-9c8b2717-05ab-4575-a08b-04568b7cfc04 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a9c2284769e842799979dfb913996f6b'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:46.262038 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664dc428-b82e-4bdd-90aa-45ac2fc59030 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:46.277808 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 965/1952] 10.222.0.46 () {58 vars in 1223 bytes} [Thu May 26 15:09:45 2022] DELETE /identity/v3/users/a9c2284769e842799979dfb913996f6b => generated 0 bytes in 521 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:09:46.282591 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2b1fcf-3f7c-4c91-b7ce-36740acbe35f None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:46.282882 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2b1fcf-3f7c-4c91-b7ce-36740acbe35f None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:46.283623 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec2b1fcf-3f7c-4c91-b7ce-36740acbe35f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:46.286331 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802aaaff-4c6e-4773-8815-762fbc8fdb26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:46.287278 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802aaaff-4c6e-4773-8815-762fbc8fdb26 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:46.287716 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802aaaff-4c6e-4773-8815-762fbc8fdb26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:46.288037 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802aaaff-4c6e-4773-8815-762fbc8fdb26 admin admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:46.288938 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802aaaff-4c6e-4773-8815-762fbc8fdb26 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:46.291380 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802aaaff-4c6e-4773-8815-762fbc8fdb26 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:46.333337 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-ec2b1fcf-3f7c-4c91-b7ce-36740acbe35f None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:09:46.334959 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-ec2b1fcf-3f7c-4c91-b7ce-36740acbe35f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:09:46.367384 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 988/1953] 10.222.0.46 () {60 vars in 1152 bytes} [Thu May 26 15:09:46 2022] POST /identity/v3/projects => generated 377 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 15:09:46.372506 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802aaaff-4c6e-4773-8815-762fbc8fdb26 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:46.447301 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:09:46.480180 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 966/1954] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:09:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 15:09:46.492422 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2b1fcf-3f7c-4c91-b7ce-36740acbe35f None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:46.494385 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-ec2b1fcf-3f7c-4c91-b7ce-36740acbe35f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:09:46.509652 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802aaaff-4c6e-4773-8815-762fbc8fdb26 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:46.516967 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5bb542-dfdf-421b-9712-9fafca83943c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:46.517995 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5bb542-dfdf-421b-9712-9fafca83943c admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:46.518360 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5bb542-dfdf-421b-9712-9fafca83943c admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:46.518761 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e5bb542-dfdf-421b-9712-9fafca83943c admin admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:46.519597 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5bb542-dfdf-421b-9712-9fafca83943c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:46.522486 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e5bb542-dfdf-421b-9712-9fafca83943c admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:46.567074 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 989/1955] 10.222.0.46 () {60 vars in 1143 bytes} [Thu May 26 15:09:46 2022] POST /identity/v3/users => generated 289 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 15:09:46.568801 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5bb542-dfdf-421b-9712-9fafca83943c admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:46.597235 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e5bb542-dfdf-421b-9712-9fafca83943c admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:46.613317 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f91d8a-40c2-4d02-bc66-8b8dbccc074e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:46.614046 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f91d8a-40c2-4d02-bc66-8b8dbccc074e admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:46.614332 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f91d8a-40c2-4d02-bc66-8b8dbccc074e admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:46.614612 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f91d8a-40c2-4d02-bc66-8b8dbccc074e admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:46.616387 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f91d8a-40c2-4d02-bc66-8b8dbccc074e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:46.618299 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f91d8a-40c2-4d02-bc66-8b8dbccc074e admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:46.632336 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 990/1956] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 15:09:46 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 15:09:46.633612 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f91d8a-40c2-4d02-bc66-8b8dbccc074e admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:46.635933 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-a4f91d8a-40c2-4d02-bc66-8b8dbccc074e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:09:46.665755 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f91d8a-40c2-4d02-bc66-8b8dbccc074e admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:46.673899 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4a89d5-33c1-4dd8-8b65-f6a23cfbeacb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:46.675696 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4a89d5-33c1-4dd8-8b65-f6a23cfbeacb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:46.676343 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4a89d5-33c1-4dd8-8b65-f6a23cfbeacb admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:46.676931 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4a89d5-33c1-4dd8-8b65-f6a23cfbeacb admin admin] PATH_INFO: `/v3/users/fd62eb0876fa4a869eb1ec531a3675be` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:46.678825 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4a89d5-33c1-4dd8-8b65-f6a23cfbeacb admin admin] RBAC: Authorizing `identity:delete_user(user_id=fd62eb0876fa4a869eb1ec531a3675be)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:46.683592 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4a89d5-33c1-4dd8-8b65-f6a23cfbeacb admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:46.691941 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec2b1fcf-3f7c-4c91-b7ce-36740acbe35f None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:46.714091 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:09:46.792859 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29c58a6-c750-4d40-af10-afd8b72c170d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0156a2f6f3794f479e76b85bdbbe5c26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:46.794017 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29c58a6-c750-4d40-af10-afd8b72c170d service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:46.794235 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29c58a6-c750-4d40-af10-afd8b72c170d service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:46.794516 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29c58a6-c750-4d40-af10-afd8b72c170d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:46.814881 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29c58a6-c750-4d40-af10-afd8b72c170d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:46.815845 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29c58a6-c750-4d40-af10-afd8b72c170d service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:46.829230 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 967/1957] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:09:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 15:09:46.830032 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29c58a6-c750-4d40-af10-afd8b72c170d service neutron] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:46.830197 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-5a4a89d5-33c1-4dd8-8b65-f6a23cfbeacb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fd62eb0876fa4a869eb1ec531a3675be'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:46.841169 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-5a4a89d5-33c1-4dd8-8b65-f6a23cfbeacb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fd62eb0876fa4a869eb1ec531a3675be'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:46.842072 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-5a4a89d5-33c1-4dd8-8b65-f6a23cfbeacb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fd62eb0876fa4a869eb1ec531a3675be'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:46.842855 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29c58a6-c750-4d40-af10-afd8b72c170d service neutron] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:46.851269 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a057aa-d192-4c7b-bf5f-35e3eb5f2842 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:46.852667 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a057aa-d192-4c7b-bf5f-35e3eb5f2842 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:46.852667 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a057aa-d192-4c7b-bf5f-35e3eb5f2842 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:46.852667 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a057aa-d192-4c7b-bf5f-35e3eb5f2842 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:46.853673 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a057aa-d192-4c7b-bf5f-35e3eb5f2842 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:46.856090 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a057aa-d192-4c7b-bf5f-35e3eb5f2842 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:46.872013 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 968/1958] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 15:09:46 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 15:09:46.876171 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a057aa-d192-4c7b-bf5f-35e3eb5f2842 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:46.878588 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-48a057aa-d192-4c7b-bf5f-35e3eb5f2842 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:09:46.879399 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-5a4a89d5-33c1-4dd8-8b65-f6a23cfbeacb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fd62eb0876fa4a869eb1ec531a3675be'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:46.896775 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-5a4a89d5-33c1-4dd8-8b65-f6a23cfbeacb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fd62eb0876fa4a869eb1ec531a3675be'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:46.897600 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a057aa-d192-4c7b-bf5f-35e3eb5f2842 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:46.905548 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 991/1959] 10.222.0.46 () {58 vars in 1223 bytes} [Thu May 26 15:09:46 2022] DELETE /identity/v3/users/fd62eb0876fa4a869eb1ec531a3675be => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:09:46.909336 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b6771e-af07-49fd-b21a-220804b673ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:46.910646 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b6771e-af07-49fd-b21a-220804b673ea admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:46.910843 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69e4072-3dde-45c3-9b8c-5a87c8b391d9 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:46.911384 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69e4072-3dde-45c3-9b8c-5a87c8b391d9 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:46.911757 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b6771e-af07-49fd-b21a-220804b673ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:46.912306 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69e4072-3dde-45c3-9b8c-5a87c8b391d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:46.912605 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b6771e-af07-49fd-b21a-220804b673ea admin admin] PATH_INFO: `/v3/projects/98699592822c4bd3ab19bc66ccfd01d1/users/5ee7f55d453648fd9af34906b217c5fb/roles/d4088d1845a24fbd9bd8975ce0b1d3d0` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:46.937201 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-e69e4072-3dde-45c3-9b8c-5a87c8b391d9 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:09:46.937976 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b6771e-af07-49fd-b21a-220804b673ea admin admin] RBAC: Authorizing `identity:create_grant(project_id=98699592822c4bd3ab19bc66ccfd01d1, user_id=5ee7f55d453648fd9af34906b217c5fb, role_id=d4088d1845a24fbd9bd8975ce0b1d3d0)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:46.939256 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-e69e4072-3dde-45c3-9b8c-5a87c8b391d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:09:46.942757 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b6771e-af07-49fd-b21a-220804b673ea admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:46.998078 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 969/1960] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 15:09:46 2022] PUT /identity/v3/projects/98699592822c4bd3ab19bc66ccfd01d1/users/5ee7f55d453648fd9af34906b217c5fb/roles/d4088d1845a24fbd9bd8975ce0b1d3d0 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:09:47.011548 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4ca03a-d9d4-48f6-8e05-3ac401b02c39 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:47.012485 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4ca03a-d9d4-48f6-8e05-3ac401b02c39 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:47.012866 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4ca03a-d9d4-48f6-8e05-3ac401b02c39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:47.061658 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 992/1961] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:09:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:09:47.082066 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-1c4ca03a-d9d4-48f6-8e05-3ac401b02c39 None None] MFA Rules not processed for user `5ee7f55d453648fd9af34906b217c5fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:09:47.083607 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-1c4ca03a-d9d4-48f6-8e05-3ac401b02c39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:09:47.088792 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69e4072-3dde-45c3-9b8c-5a87c8b391d9 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:47.093811 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-e69e4072-3dde-45c3-9b8c-5a87c8b391d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:09:47.117713 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:09:47.202683 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69e4072-3dde-45c3-9b8c-5a87c8b391d9 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:47.259658 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 970/1962] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:09:47 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:09:47.278110 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4ca03a-d9d4-48f6-8e05-3ac401b02c39 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:47.278110 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-1c4ca03a-d9d4-48f6-8e05-3ac401b02c39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:09:47.339439 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9704d857-7361-4b0b-ad5a-22b56d1e2853 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0156a2f6f3794f479e76b85bdbbe5c26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:47.340796 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9704d857-7361-4b0b-ad5a-22b56d1e2853 service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:47.341536 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9704d857-7361-4b0b-ad5a-22b56d1e2853 service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:47.342372 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9704d857-7361-4b0b-ad5a-22b56d1e2853 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:47.430646 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4ca03a-d9d4-48f6-8e05-3ac401b02c39 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:47.430646 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3fb801-de34-4cc2-8c07-9cd239cd6693 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5b5b35614a547d783e6e77ddb8c1c03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:47.430834 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3fb801-de34-4cc2-8c07-9cd239cd6693 service cinder] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:47.431027 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3fb801-de34-4cc2-8c07-9cd239cd6693 service cinder] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:47.431312 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3fb801-de34-4cc2-8c07-9cd239cd6693 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:47.459382 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3fb801-de34-4cc2-8c07-9cd239cd6693 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:47.461926 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3fb801-de34-4cc2-8c07-9cd239cd6693 service cinder] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:47.477579 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9704d857-7361-4b0b-ad5a-22b56d1e2853 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:47.480627 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9704d857-7361-4b0b-ad5a-22b56d1e2853 service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:47.484265 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 971/1963] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:09:47 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 15:09:47.525427 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 993/1964] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:09:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 438 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 15:09:47.526926 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3fb801-de34-4cc2-8c07-9cd239cd6693 service cinder] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:47.529743 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-ff3fb801-de34-4cc2-8c07-9cd239cd6693 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:09:47.551667 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3fb801-de34-4cc2-8c07-9cd239cd6693 service cinder] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:47.557426 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d4dbb8-7e94-462c-9cb5-1609ec10fcb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:47.558583 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d4dbb8-7e94-462c-9cb5-1609ec10fcb7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:47.559384 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d4dbb8-7e94-462c-9cb5-1609ec10fcb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:47.559665 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d4dbb8-7e94-462c-9cb5-1609ec10fcb7 admin admin] PATH_INFO: `/v3/projects/1927b6f4b6d44a6088c33e8c0d30cd3b` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:47.561238 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d4dbb8-7e94-462c-9cb5-1609ec10fcb7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1927b6f4b6d44a6088c33e8c0d30cd3b)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:47.563437 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d4dbb8-7e94-462c-9cb5-1609ec10fcb7 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:47.580115 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:09:47.656877 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-65d4dbb8-7e94-462c-9cb5-1609ec10fcb7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1927b6f4b6d44a6088c33e8c0d30cd3b', 'request_id': 'req-65d4dbb8-7e94-462c-9cb5-1609ec10fcb7'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:47.668993 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-65d4dbb8-7e94-462c-9cb5-1609ec10fcb7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1927b6f4b6d44a6088c33e8c0d30cd3b', 'request_id': 'req-65d4dbb8-7e94-462c-9cb5-1609ec10fcb7'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:47.691462 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-65d4dbb8-7e94-462c-9cb5-1609ec10fcb7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1927b6f4b6d44a6088c33e8c0d30cd3b', 'request_id': 'req-65d4dbb8-7e94-462c-9cb5-1609ec10fcb7'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:47.713891 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-65d4dbb8-7e94-462c-9cb5-1609ec10fcb7 admin admin] The token cache is being invalidate because project 1927b6f4b6d44a6088c33e8c0d30cd3b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 15:09:47.714265 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-65d4dbb8-7e94-462c-9cb5-1609ec10fcb7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:47.715102 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-65d4dbb8-7e94-462c-9cb5-1609ec10fcb7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:47.718884 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 972/1965] 10.222.0.46 () {58 vars in 1232 bytes} [Thu May 26 15:09:47 2022] DELETE /identity/v3/projects/1927b6f4b6d44a6088c33e8c0d30cd3b => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:09:47.723857 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9704d857-7361-4b0b-ad5a-22b56d1e2853 service neutron] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:47.742147 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9704d857-7361-4b0b-ad5a-22b56d1e2853 service neutron] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:47.861564 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee121b61-8ed6-4096-93c2-030cc8c0f4c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:47.862652 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee121b61-8ed6-4096-93c2-030cc8c0f4c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:47.862958 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee121b61-8ed6-4096-93c2-030cc8c0f4c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:47.863240 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee121b61-8ed6-4096-93c2-030cc8c0f4c0 admin admin] PATH_INFO: `/v3/users/818797c803554fe292ad3b8ab6f90fbf` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:47.864611 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee121b61-8ed6-4096-93c2-030cc8c0f4c0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=818797c803554fe292ad3b8ab6f90fbf)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:47.866895 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee121b61-8ed6-4096-93c2-030cc8c0f4c0 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:47.902036 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:09:47.961708 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-ee121b61-8ed6-4096-93c2-030cc8c0f4c0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '818797c803554fe292ad3b8ab6f90fbf'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:47.987177 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-ee121b61-8ed6-4096-93c2-030cc8c0f4c0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '818797c803554fe292ad3b8ab6f90fbf'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:47.987177 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-ee121b61-8ed6-4096-93c2-030cc8c0f4c0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '818797c803554fe292ad3b8ab6f90fbf'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:48.010305 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-ee121b61-8ed6-4096-93c2-030cc8c0f4c0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '818797c803554fe292ad3b8ab6f90fbf'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:48.030215 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-ee121b61-8ed6-4096-93c2-030cc8c0f4c0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '818797c803554fe292ad3b8ab6f90fbf'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:48.041203 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 994/1966] 10.222.0.46 () {58 vars in 1223 bytes} [Thu May 26 15:09:47 2022] DELETE /identity/v3/users/818797c803554fe292ad3b8ab6f90fbf => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:09:48.590131 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d4dbb8-7e94-462c-9cb5-1609ec10fcb7 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:48.591811 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-65d4dbb8-7e94-462c-9cb5-1609ec10fcb7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:09:48.626689 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d4dbb8-7e94-462c-9cb5-1609ec10fcb7 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:48.724881 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf2eb6f-48cb-4bdd-a208-a8432115e248 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:48.726801 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf2eb6f-48cb-4bdd-a208-a8432115e248 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:48.727081 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf2eb6f-48cb-4bdd-a208-a8432115e248 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:48.727360 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf2eb6f-48cb-4bdd-a208-a8432115e248 admin admin] PATH_INFO: `/v3/projects/bdc9f8ca84bb4d33b2c7f76ab47f58d0` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:48.728902 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf2eb6f-48cb-4bdd-a208-a8432115e248 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bdc9f8ca84bb4d33b2c7f76ab47f58d0)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:48.732591 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf2eb6f-48cb-4bdd-a208-a8432115e248 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:48.788867 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee121b61-8ed6-4096-93c2-030cc8c0f4c0 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:48.791938 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-ee121b61-8ed6-4096-93c2-030cc8c0f4c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:09:48.804387 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-2cf2eb6f-48cb-4bdd-a208-a8432115e248 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bdc9f8ca84bb4d33b2c7f76ab47f58d0', 'request_id': 'req-2cf2eb6f-48cb-4bdd-a208-a8432115e248'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:48.813613 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-2cf2eb6f-48cb-4bdd-a208-a8432115e248 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bdc9f8ca84bb4d33b2c7f76ab47f58d0', 'request_id': 'req-2cf2eb6f-48cb-4bdd-a208-a8432115e248'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:48.825564 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-2cf2eb6f-48cb-4bdd-a208-a8432115e248 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bdc9f8ca84bb4d33b2c7f76ab47f58d0', 'request_id': 'req-2cf2eb6f-48cb-4bdd-a208-a8432115e248'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:48.825859 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee121b61-8ed6-4096-93c2-030cc8c0f4c0 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:48.851989 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-2cf2eb6f-48cb-4bdd-a208-a8432115e248 admin admin] The token cache is being invalidate because project bdc9f8ca84bb4d33b2c7f76ab47f58d0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 15:09:48.852450 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-2cf2eb6f-48cb-4bdd-a208-a8432115e248 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:48.853440 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-2cf2eb6f-48cb-4bdd-a208-a8432115e248 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:48.857109 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 973/1967] 10.222.0.46 () {58 vars in 1232 bytes} [Thu May 26 15:09:48 2022] DELETE /identity/v3/projects/bdc9f8ca84bb4d33b2c7f76ab47f58d0 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:09:48.862931 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf2eb6f-48cb-4bdd-a208-a8432115e248 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:48.865383 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-2cf2eb6f-48cb-4bdd-a208-a8432115e248 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:09:48.914480 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:09:48.915841 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356781f3-7053-4f56-b31f-a897cedf4e31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:48.916848 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356781f3-7053-4f56-b31f-a897cedf4e31 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:48.917342 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356781f3-7053-4f56-b31f-a897cedf4e31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:48.917696 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356781f3-7053-4f56-b31f-a897cedf4e31 admin admin] PATH_INFO: `/v3/projects/d0ab6453132948038ccb7f4e8176e5e7` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:48.919882 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356781f3-7053-4f56-b31f-a897cedf4e31 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d0ab6453132948038ccb7f4e8176e5e7)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:48.923168 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356781f3-7053-4f56-b31f-a897cedf4e31 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:48.981509 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf2eb6f-48cb-4bdd-a208-a8432115e248 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:48.994057 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d8484a-0a8f-4b7c-8751-4db46eb52c90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:49.008747 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d8484a-0a8f-4b7c-8751-4db46eb52c90 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:49.009046 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d8484a-0a8f-4b7c-8751-4db46eb52c90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:49.009325 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d8484a-0a8f-4b7c-8751-4db46eb52c90 admin admin] PATH_INFO: `/v3/users/bbb01d7afc0e4ce4a17a777bf6bea3ad` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:49.015571 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d8484a-0a8f-4b7c-8751-4db46eb52c90 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bbb01d7afc0e4ce4a17a777bf6bea3ad)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:49.019388 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d8484a-0a8f-4b7c-8751-4db46eb52c90 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:49.044657 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-356781f3-7053-4f56-b31f-a897cedf4e31 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd0ab6453132948038ccb7f4e8176e5e7', 'request_id': 'req-356781f3-7053-4f56-b31f-a897cedf4e31'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:49.066156 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-356781f3-7053-4f56-b31f-a897cedf4e31 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd0ab6453132948038ccb7f4e8176e5e7', 'request_id': 'req-356781f3-7053-4f56-b31f-a897cedf4e31'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:49.091572 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-356781f3-7053-4f56-b31f-a897cedf4e31 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd0ab6453132948038ccb7f4e8176e5e7', 'request_id': 'req-356781f3-7053-4f56-b31f-a897cedf4e31'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:49.107913 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-65d8484a-0a8f-4b7c-8751-4db46eb52c90 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bbb01d7afc0e4ce4a17a777bf6bea3ad'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:49.122100 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-65d8484a-0a8f-4b7c-8751-4db46eb52c90 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bbb01d7afc0e4ce4a17a777bf6bea3ad'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:49.124084 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-65d8484a-0a8f-4b7c-8751-4db46eb52c90 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bbb01d7afc0e4ce4a17a777bf6bea3ad'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:49.142244 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-356781f3-7053-4f56-b31f-a897cedf4e31 admin admin] The token cache is being invalidate because project d0ab6453132948038ccb7f4e8176e5e7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 15:09:49.142665 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-356781f3-7053-4f56-b31f-a897cedf4e31 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:49.144648 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-356781f3-7053-4f56-b31f-a897cedf4e31 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:49.161349 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 995/1968] 10.222.0.46 () {58 vars in 1232 bytes} [Thu May 26 15:09:48 2022] DELETE /identity/v3/projects/d0ab6453132948038ccb7f4e8176e5e7 => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 15:09:49.161349 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356781f3-7053-4f56-b31f-a897cedf4e31 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:49.162732 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-356781f3-7053-4f56-b31f-a897cedf4e31 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:09:49.165997 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-65d8484a-0a8f-4b7c-8751-4db46eb52c90 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bbb01d7afc0e4ce4a17a777bf6bea3ad'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:49.189785 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-65d8484a-0a8f-4b7c-8751-4db46eb52c90 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bbb01d7afc0e4ce4a17a777bf6bea3ad'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:49.203610 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 974/1969] 10.222.0.46 () {58 vars in 1223 bytes} [Thu May 26 15:09:48 2022] DELETE /identity/v3/users/bbb01d7afc0e4ce4a17a777bf6bea3ad => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 15:09:49.205711 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d53f3f6-f6cf-4976-adc4-b78b18c4e1ce None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:49.206083 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d53f3f6-f6cf-4976-adc4-b78b18c4e1ce None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:49.206449 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d53f3f6-f6cf-4976-adc4-b78b18c4e1ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:49.240559 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:09:49.256455 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-1d53f3f6-f6cf-4976-adc4-b78b18c4e1ce None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:09:49.257985 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-1d53f3f6-f6cf-4976-adc4-b78b18c4e1ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:09:49.325092 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356781f3-7053-4f56-b31f-a897cedf4e31 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:49.437887 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63307d4a-c2a9-4a08-aef2-5e1b4021b89a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:49.441632 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63307d4a-c2a9-4a08-aef2-5e1b4021b89a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:49.441939 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63307d4a-c2a9-4a08-aef2-5e1b4021b89a admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:49.442225 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63307d4a-c2a9-4a08-aef2-5e1b4021b89a admin admin] PATH_INFO: `/v3/users/79615a15f7634b1fbfa2172227b2d39b` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:49.443817 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63307d4a-c2a9-4a08-aef2-5e1b4021b89a admin admin] RBAC: Authorizing `identity:delete_user(user_id=79615a15f7634b1fbfa2172227b2d39b)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:49.450003 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63307d4a-c2a9-4a08-aef2-5e1b4021b89a admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:49.517929 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 975/1970] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:09:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 314 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:09:49.528208 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58faaeb3-0650-4d61-a6d4-171e7e651dd4 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:49.528653 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58faaeb3-0650-4d61-a6d4-171e7e651dd4 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:49.529096 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58faaeb3-0650-4d61-a6d4-171e7e651dd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:49.598339 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-58faaeb3-0650-4d61-a6d4-171e7e651dd4 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:09:49.599883 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-58faaeb3-0650-4d61-a6d4-171e7e651dd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:09:49.600362 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-63307d4a-c2a9-4a08-aef2-5e1b4021b89a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '79615a15f7634b1fbfa2172227b2d39b'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:49.619923 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 976/1971] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:09:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 15:09:49.628921 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-63307d4a-c2a9-4a08-aef2-5e1b4021b89a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '79615a15f7634b1fbfa2172227b2d39b'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:49.628921 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-63307d4a-c2a9-4a08-aef2-5e1b4021b89a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '79615a15f7634b1fbfa2172227b2d39b'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:49.634178 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58faaeb3-0650-4d61-a6d4-171e7e651dd4 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:49.656432 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58faaeb3-0650-4d61-a6d4-171e7e651dd4 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:49.662730 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b77744c-4878-418e-9d6a-a7582678346d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:49.663928 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b77744c-4878-418e-9d6a-a7582678346d admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:49.664801 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b77744c-4878-418e-9d6a-a7582678346d admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:49.664967 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-63307d4a-c2a9-4a08-aef2-5e1b4021b89a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '79615a15f7634b1fbfa2172227b2d39b'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:49.665720 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b77744c-4878-418e-9d6a-a7582678346d admin admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:49.666748 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b77744c-4878-418e-9d6a-a7582678346d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 15:09:49.667304 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b77744c-4878-418e-9d6a-a7582678346d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:49.670331 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b77744c-4878-418e-9d6a-a7582678346d admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:49.687236 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 977/1972] 10.222.0.46 () {58 vars in 1152 bytes} [Thu May 26 15:09:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 15:09:49.688596 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-63307d4a-c2a9-4a08-aef2-5e1b4021b89a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '79615a15f7634b1fbfa2172227b2d39b'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:49.691505 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b77744c-4878-418e-9d6a-a7582678346d admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:49.695207 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-3b77744c-4878-418e-9d6a-a7582678346d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:09:49.706553 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 996/1973] 10.222.0.46 () {58 vars in 1223 bytes} [Thu May 26 15:09:49 2022] DELETE /identity/v3/users/79615a15f7634b1fbfa2172227b2d39b => generated 0 bytes in 546 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:09:49.708083 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b44759-367b-4f07-a7b2-3ae2ec593e16 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:49.708403 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b44759-367b-4f07-a7b2-3ae2ec593e16 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:49.708697 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b44759-367b-4f07-a7b2-3ae2ec593e16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:49.728412 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b77744c-4878-418e-9d6a-a7582678346d admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:49.752833 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-09b44759-367b-4f07-a7b2-3ae2ec593e16 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:09:49.754133 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-09b44759-367b-4f07-a7b2-3ae2ec593e16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:09:49.803570 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:09:49.891950 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:09:49.919269 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8434ac-5d57-498c-badc-a110b69d8e3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:49.924348 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8434ac-5d57-498c-badc-a110b69d8e3a admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:49.924874 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8434ac-5d57-498c-badc-a110b69d8e3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:49.926281 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c8434ac-5d57-498c-badc-a110b69d8e3a admin admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:49.927189 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8434ac-5d57-498c-badc-a110b69d8e3a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:49.932827 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 997/1974] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:09:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:09:49.933016 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c8434ac-5d57-498c-badc-a110b69d8e3a admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:49.967578 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b44759-367b-4f07-a7b2-3ae2ec593e16 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:49.970052 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-09b44759-367b-4f07-a7b2-3ae2ec593e16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:09:50.013941 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 978/1975] 10.222.0.46 () {60 vars in 1152 bytes} [Thu May 26 15:09:49 2022] POST /identity/v3/projects => generated 399 bytes in 323 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 15:09:50.017416 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8434ac-5d57-498c-badc-a110b69d8e3a admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:50.114581 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b44759-367b-4f07-a7b2-3ae2ec593e16 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:50.185704 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c8434ac-5d57-498c-badc-a110b69d8e3a admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:50.191775 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ca2915-b425-4dd2-8a2c-968972b966ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:50.193430 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ca2915-b425-4dd2-8a2c-968972b966ca admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:50.193786 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ca2915-b425-4dd2-8a2c-968972b966ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:50.194486 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ca2915-b425-4dd2-8a2c-968972b966ca admin admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:50.195371 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ca2915-b425-4dd2-8a2c-968972b966ca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:50.198155 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ca2915-b425-4dd2-8a2c-968972b966ca admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:50.239447 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 979/1976] 10.222.0.46 () {60 vars in 1143 bytes} [Thu May 26 15:09:50 2022] POST /identity/v3/users => generated 300 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 15:09:50.240671 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ca2915-b425-4dd2-8a2c-968972b966ca admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:50.245140 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-33ca2915-b425-4dd2-8a2c-968972b966ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:09:50.263040 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a37b533-368f-4208-ae2a-edfeeab8b43e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0156a2f6f3794f479e76b85bdbbe5c26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:50.267879 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a37b533-368f-4208-ae2a-edfeeab8b43e service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:50.268410 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a37b533-368f-4208-ae2a-edfeeab8b43e service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:50.268722 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a37b533-368f-4208-ae2a-edfeeab8b43e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:50.291938 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ca2915-b425-4dd2-8a2c-968972b966ca admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:50.298905 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ceda9f-35ca-44f4-8956-bcf79ca44053 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:50.300504 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ceda9f-35ca-44f4-8956-bcf79ca44053 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:50.301256 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ceda9f-35ca-44f4-8956-bcf79ca44053 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:50.301975 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ceda9f-35ca-44f4-8956-bcf79ca44053 admin admin] PATH_INFO: `/v3/projects/658b0d6c1e9b46fe981657f449ef778e` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:50.303636 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ceda9f-35ca-44f4-8956-bcf79ca44053 admin admin] RBAC: Authorizing `identity:delete_project(project_id=658b0d6c1e9b46fe981657f449ef778e)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:50.306375 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ceda9f-35ca-44f4-8956-bcf79ca44053 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:50.307759 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a37b533-368f-4208-ae2a-edfeeab8b43e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:50.310120 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a37b533-368f-4208-ae2a-edfeeab8b43e service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:50.335338 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 998/1977] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:09:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 15:09:50.336217 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a37b533-368f-4208-ae2a-edfeeab8b43e service neutron] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:50.348954 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a37b533-368f-4208-ae2a-edfeeab8b43e service neutron] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:50.368205 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a208f9-cdd7-42a2-8c61-ae442e4987ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:50.368894 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a208f9-cdd7-42a2-8c61-ae442e4987ed admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:50.369297 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a208f9-cdd7-42a2-8c61-ae442e4987ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:50.369676 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a208f9-cdd7-42a2-8c61-ae442e4987ed admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:50.370554 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a208f9-cdd7-42a2-8c61-ae442e4987ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:50.375521 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a208f9-cdd7-42a2-8c61-ae442e4987ed admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:50.395983 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 999/1978] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 15:09:50 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 15:09:50.402785 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a208f9-cdd7-42a2-8c61-ae442e4987ed admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:50.418272 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a208f9-cdd7-42a2-8c61-ae442e4987ed admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:50.441810 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-28ceda9f-35ca-44f4-8956-bcf79ca44053 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '658b0d6c1e9b46fe981657f449ef778e', 'request_id': 'req-28ceda9f-35ca-44f4-8956-bcf79ca44053'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:50.454571 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-28ceda9f-35ca-44f4-8956-bcf79ca44053 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '658b0d6c1e9b46fe981657f449ef778e', 'request_id': 'req-28ceda9f-35ca-44f4-8956-bcf79ca44053'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:50.461988 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-28ceda9f-35ca-44f4-8956-bcf79ca44053 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '658b0d6c1e9b46fe981657f449ef778e', 'request_id': 'req-28ceda9f-35ca-44f4-8956-bcf79ca44053'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:50.479638 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-28ceda9f-35ca-44f4-8956-bcf79ca44053 admin admin] The token cache is being invalidate because project 658b0d6c1e9b46fe981657f449ef778e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 15:09:50.479914 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-28ceda9f-35ca-44f4-8956-bcf79ca44053 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:50.481242 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-28ceda9f-35ca-44f4-8956-bcf79ca44053 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:50.488007 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 980/1979] 10.222.0.46 () {58 vars in 1232 bytes} [Thu May 26 15:09:50 2022] DELETE /identity/v3/projects/658b0d6c1e9b46fe981657f449ef778e => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:09:50.490871 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ea49ec-d461-473a-b46f-c9ec28a562ad None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:50.491214 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ea49ec-d461-473a-b46f-c9ec28a562ad None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:50.491471 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ea49ec-d461-473a-b46f-c9ec28a562ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:50.517642 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abb5073-d409-4e8a-a88f-4573e438a0fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:50.518401 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abb5073-d409-4e8a-a88f-4573e438a0fa admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:50.518729 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abb5073-d409-4e8a-a88f-4573e438a0fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:50.518983 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abb5073-d409-4e8a-a88f-4573e438a0fa admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:50.519728 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abb5073-d409-4e8a-a88f-4573e438a0fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:50.522281 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abb5073-d409-4e8a-a88f-4573e438a0fa admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:50.523810 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-e8ea49ec-d461-473a-b46f-c9ec28a562ad None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:09:50.525354 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-e8ea49ec-d461-473a-b46f-c9ec28a562ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:09:50.546734 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 981/1980] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:09:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:09:50.546954 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1000/1981] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 15:09:50 2022] GET /identity/v3/roles => generated 1420 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 15:09:50.549442 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ea49ec-d461-473a-b46f-c9ec28a562ad None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:50.550042 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6874696-2b06-4745-af3f-1966e71f0ac7 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:50.550273 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6874696-2b06-4745-af3f-1966e71f0ac7 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:50.550820 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6874696-2b06-4745-af3f-1966e71f0ac7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:50.551714 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-e8ea49ec-d461-473a-b46f-c9ec28a562ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:09:50.568029 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8ea49ec-d461-473a-b46f-c9ec28a562ad None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:50.574678 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782c208d-3509-444b-831b-f9ddbd9187cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:50.575673 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782c208d-3509-444b-831b-f9ddbd9187cf admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:50.576243 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782c208d-3509-444b-831b-f9ddbd9187cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:50.576362 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782c208d-3509-444b-831b-f9ddbd9187cf admin admin] PATH_INFO: `/v3/projects/6f563cb6ba894aab9ad9aa10193a6bd6/users/0425558227fb4feeba1898602aeefba6/roles/d4088d1845a24fbd9bd8975ce0b1d3d0` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:50.580892 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-a6874696-2b06-4745-af3f-1966e71f0ac7 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:09:50.582524 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-a6874696-2b06-4745-af3f-1966e71f0ac7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:09:50.598570 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1001/1982] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:09:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 15:09:50.602863 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6874696-2b06-4745-af3f-1966e71f0ac7 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:50.603083 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782c208d-3509-444b-831b-f9ddbd9187cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f563cb6ba894aab9ad9aa10193a6bd6, user_id=0425558227fb4feeba1898602aeefba6, role_id=d4088d1845a24fbd9bd8975ce0b1d3d0)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:50.609546 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782c208d-3509-444b-831b-f9ddbd9187cf admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:50.619134 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=87,keystone:INSERT=6,keystone:DELETE=7 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:09:50.619332 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6874696-2b06-4745-af3f-1966e71f0ac7 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:50.636448 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 982/1983] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 15:09:50 2022] PUT /identity/v3/projects/6f563cb6ba894aab9ad9aa10193a6bd6/users/0425558227fb4feeba1898602aeefba6/roles/d4088d1845a24fbd9bd8975ce0b1d3d0 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 15:09:50.641589 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65d4268-7873-4b53-af26-16705ce0b226 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:50.642701 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65d4268-7873-4b53-af26-16705ce0b226 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:50.642990 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65d4268-7873-4b53-af26-16705ce0b226 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:50.643406 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65d4268-7873-4b53-af26-16705ce0b226 admin admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:50.644433 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65d4268-7873-4b53-af26-16705ce0b226 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 15:09:50.644753 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65d4268-7873-4b53-af26-16705ce0b226 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:50.646888 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65d4268-7873-4b53-af26-16705ce0b226 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:50.652847 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0155de6a-1b63-4df7-967b-40b1f6e51821 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:50.653101 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0155de6a-1b63-4df7-967b-40b1f6e51821 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:50.655683 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0155de6a-1b63-4df7-967b-40b1f6e51821 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:50.668126 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1002/1984] 10.222.0.46 () {58 vars in 1152 bytes} [Thu May 26 15:09:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 15:09:50.678534 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65d4268-7873-4b53-af26-16705ce0b226 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:50.696836 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65d4268-7873-4b53-af26-16705ce0b226 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:50.703675 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-0155de6a-1b63-4df7-967b-40b1f6e51821 None None] MFA Rules not processed for user `0425558227fb4feeba1898602aeefba6`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:09:50.706563 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-0155de6a-1b63-4df7-967b-40b1f6e51821 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:09:50.722185 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:09:50.776595 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fcc5ed-11a0-4ba7-a830-3024587c0497 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:50.778035 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fcc5ed-11a0-4ba7-a830-3024587c0497 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:50.779166 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fcc5ed-11a0-4ba7-a830-3024587c0497 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:50.779535 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fcc5ed-11a0-4ba7-a830-3024587c0497 admin admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:50.780332 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fcc5ed-11a0-4ba7-a830-3024587c0497 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:50.782860 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fcc5ed-11a0-4ba7-a830-3024587c0497 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:50.830354 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1003/1985] 10.222.0.46 () {60 vars in 1152 bytes} [Thu May 26 15:09:50 2022] POST /identity/v3/projects => generated 373 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 15:09:50.831327 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fcc5ed-11a0-4ba7-a830-3024587c0497 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:50.837194 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 983/1986] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:09:50 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:09:50.839594 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0155de6a-1b63-4df7-967b-40b1f6e51821 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:50.862265 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0155de6a-1b63-4df7-967b-40b1f6e51821 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:50.946522 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fcc5ed-11a0-4ba7-a830-3024587c0497 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:50.955429 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b996d130-6517-4254-86d1-4e21b90ca75c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:50.956647 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b996d130-6517-4254-86d1-4e21b90ca75c admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:50.956790 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b996d130-6517-4254-86d1-4e21b90ca75c admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:50.957056 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b996d130-6517-4254-86d1-4e21b90ca75c admin admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:50.957801 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b996d130-6517-4254-86d1-4e21b90ca75c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:50.960082 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b996d130-6517-4254-86d1-4e21b90ca75c admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:50.990433 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1004/1987] 10.222.0.46 () {60 vars in 1143 bytes} [Thu May 26 15:09:50 2022] POST /identity/v3/users => generated 287 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 15:09:50.991699 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9f5ddb-a7fe-46f3-9963-8147883bb8df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:50.993298 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9f5ddb-a7fe-46f3-9963-8147883bb8df admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:50.993796 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9f5ddb-a7fe-46f3-9963-8147883bb8df admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:50.994180 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9f5ddb-a7fe-46f3-9963-8147883bb8df admin admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:50.995067 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9f5ddb-a7fe-46f3-9963-8147883bb8df admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:50.995834 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b996d130-6517-4254-86d1-4e21b90ca75c admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:50.999942 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9f5ddb-a7fe-46f3-9963-8147883bb8df admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:51.012727 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b996d130-6517-4254-86d1-4e21b90ca75c admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:51.020540 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f130037-ceae-444a-88d1-da0caec31f9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:51.021627 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f130037-ceae-444a-88d1-da0caec31f9a admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:51.022016 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f130037-ceae-444a-88d1-da0caec31f9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:51.022554 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f130037-ceae-444a-88d1-da0caec31f9a admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:51.023643 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f130037-ceae-444a-88d1-da0caec31f9a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:51.025362 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f130037-ceae-444a-88d1-da0caec31f9a admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:51.038574 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1005/1988] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 15:09:50 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 15:09:51.039954 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f130037-ceae-444a-88d1-da0caec31f9a admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:51.044379 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-5f130037-ceae-444a-88d1-da0caec31f9a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:09:51.051821 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 984/1989] 10.222.0.46 () {60 vars in 1152 bytes} [Thu May 26 15:09:50 2022] POST /identity/v3/projects => generated 401 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 15:09:51.051821 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9f5ddb-a7fe-46f3-9963-8147883bb8df admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:51.065690 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f130037-ceae-444a-88d1-da0caec31f9a admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:51.139421 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac7c21c-455d-4c45-b48b-0348129d4d16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:51.140629 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac7c21c-455d-4c45-b48b-0348129d4d16 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:51.144900 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac7c21c-455d-4c45-b48b-0348129d4d16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:51.145371 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac7c21c-455d-4c45-b48b-0348129d4d16 admin admin] PATH_INFO: `/v3/projects/156f43da4ee5406abc05ecc0d89e9042` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:51.146833 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac7c21c-455d-4c45-b48b-0348129d4d16 admin admin] RBAC: Authorizing `identity:delete_project(project_id=156f43da4ee5406abc05ecc0d89e9042)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:51.150601 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac7c21c-455d-4c45-b48b-0348129d4d16 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:51.159811 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9f5ddb-a7fe-46f3-9963-8147883bb8df admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:51.164828 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e460f65b-2f4b-4084-a988-422764cc353b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:51.165695 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e460f65b-2f4b-4084-a988-422764cc353b admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:51.166083 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e460f65b-2f4b-4084-a988-422764cc353b admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:51.167032 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e460f65b-2f4b-4084-a988-422764cc353b admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:51.168349 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e460f65b-2f4b-4084-a988-422764cc353b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:51.170789 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e460f65b-2f4b-4084-a988-422764cc353b admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:51.184732 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 985/1990] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 15:09:51 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 15:09:51.185770 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e460f65b-2f4b-4084-a988-422764cc353b admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:51.198992 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e460f65b-2f4b-4084-a988-422764cc353b admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:51.204418 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:09:51.204839 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e53b88a-0786-4080-b6c3-eb1325612baf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:51.207174 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e53b88a-0786-4080-b6c3-eb1325612baf admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:51.207732 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e53b88a-0786-4080-b6c3-eb1325612baf admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:51.208085 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e53b88a-0786-4080-b6c3-eb1325612baf admin admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:51.208998 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e53b88a-0786-4080-b6c3-eb1325612baf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:51.211401 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e53b88a-0786-4080-b6c3-eb1325612baf admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:51.219104 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:09:51.237534 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-3ac7c21c-455d-4c45-b48b-0348129d4d16 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '156f43da4ee5406abc05ecc0d89e9042', 'request_id': 'req-3ac7c21c-455d-4c45-b48b-0348129d4d16'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:51.243723 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 986/1991] 10.222.0.46 () {60 vars in 1143 bytes} [Thu May 26 15:09:51 2022] POST /identity/v3/users => generated 307 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 15:09:51.244895 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e53b88a-0786-4080-b6c3-eb1325612baf admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:51.254977 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-3ac7c21c-455d-4c45-b48b-0348129d4d16 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '156f43da4ee5406abc05ecc0d89e9042', 'request_id': 'req-3ac7c21c-455d-4c45-b48b-0348129d4d16'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:51.260625 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e53b88a-0786-4080-b6c3-eb1325612baf admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:51.263387 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-3ac7c21c-455d-4c45-b48b-0348129d4d16 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '156f43da4ee5406abc05ecc0d89e9042', 'request_id': 'req-3ac7c21c-455d-4c45-b48b-0348129d4d16'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:51.291171 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-3ac7c21c-455d-4c45-b48b-0348129d4d16 admin admin] The token cache is being invalidate because project 156f43da4ee5406abc05ecc0d89e9042 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 15:09:51.291754 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-3ac7c21c-455d-4c45-b48b-0348129d4d16 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:51.292748 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-3ac7c21c-455d-4c45-b48b-0348129d4d16 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:51.295347 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1006/1992] 10.222.0.46 () {58 vars in 1232 bytes} [Thu May 26 15:09:51 2022] DELETE /identity/v3/projects/156f43da4ee5406abc05ecc0d89e9042 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:09:51.296668 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac7c21c-455d-4c45-b48b-0348129d4d16 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:51.299606 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-3ac7c21c-455d-4c45-b48b-0348129d4d16 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:09:51.354658 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd63d74-36a3-417a-96f1-aff67fc66164 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:51.358196 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd63d74-36a3-417a-96f1-aff67fc66164 admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:51.358485 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd63d74-36a3-417a-96f1-aff67fc66164 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:51.358768 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd63d74-36a3-417a-96f1-aff67fc66164 admin admin] PATH_INFO: `/v3/projects/75b35b9ac1764016b396142a41ee5d1e/users/ec18059e0f2943769ae56fbf7a94be6b/roles/d4088d1845a24fbd9bd8975ce0b1d3d0` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:51.391663 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd63d74-36a3-417a-96f1-aff67fc66164 admin admin] RBAC: Authorizing `identity:create_grant(project_id=75b35b9ac1764016b396142a41ee5d1e, user_id=ec18059e0f2943769ae56fbf7a94be6b, role_id=d4088d1845a24fbd9bd8975ce0b1d3d0)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:51.395032 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac7c21c-455d-4c45-b48b-0348129d4d16 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:51.397295 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd63d74-36a3-417a-96f1-aff67fc66164 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:51.401405 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36a0558-c24c-4818-aaf6-f5cd672baacc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:51.402486 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36a0558-c24c-4818-aaf6-f5cd672baacc admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:51.403165 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36a0558-c24c-4818-aaf6-f5cd672baacc admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:51.403603 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36a0558-c24c-4818-aaf6-f5cd672baacc admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:51.404552 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36a0558-c24c-4818-aaf6-f5cd672baacc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:51.407140 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36a0558-c24c-4818-aaf6-f5cd672baacc admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:51.417601 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 987/1993] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 15:09:51 2022] PUT /identity/v3/projects/75b35b9ac1764016b396142a41ee5d1e/users/ec18059e0f2943769ae56fbf7a94be6b/roles/d4088d1845a24fbd9bd8975ce0b1d3d0 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:09:51.419127 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd63d74-36a3-417a-96f1-aff67fc66164 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:51.422002 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-fbd63d74-36a3-417a-96f1-aff67fc66164 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:09:51.425880 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1007/1994] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 15:09:51 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 15:09:51.425880 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6991e3-3429-4b34-b09b-c5e9fdf6ce5b None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:51.425880 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6991e3-3429-4b34-b09b-c5e9fdf6ce5b None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:51.426704 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6991e3-3429-4b34-b09b-c5e9fdf6ce5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:51.455388 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-bd6991e3-3429-4b34-b09b-c5e9fdf6ce5b None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:09:51.456713 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-bd6991e3-3429-4b34-b09b-c5e9fdf6ce5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:09:51.510873 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd63d74-36a3-417a-96f1-aff67fc66164 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:51.515800 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c904f4-7c03-4ce5-9c60-a695f2753146 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:51.516884 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c904f4-7c03-4ce5-9c60-a695f2753146 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:51.517199 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c904f4-7c03-4ce5-9c60-a695f2753146 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:51.517584 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c904f4-7c03-4ce5-9c60-a695f2753146 admin admin] PATH_INFO: `/v3/users/6fdcfb22d5be47dc9215d4cb0671efb4` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:51.518882 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c904f4-7c03-4ce5-9c60-a695f2753146 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6fdcfb22d5be47dc9215d4cb0671efb4)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:51.520993 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c904f4-7c03-4ce5-9c60-a695f2753146 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:51.538141 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1008/1995] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:09:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:09:51.538881 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6991e3-3429-4b34-b09b-c5e9fdf6ce5b None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:51.552261 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd6991e3-3429-4b34-b09b-c5e9fdf6ce5b None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:51.561322 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d99e7b-c03c-4a8b-924e-e88a2212dcb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:51.563026 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d99e7b-c03c-4a8b-924e-e88a2212dcb6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:51.563485 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d99e7b-c03c-4a8b-924e-e88a2212dcb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:51.563946 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d99e7b-c03c-4a8b-924e-e88a2212dcb6 admin admin] PATH_INFO: `/v3/projects/26410922fe5847ddb10de90af2b5f3d3/users/68edfbc789a648d7b0d604843f322b15/roles/a2e551773cbd4df084020676865b8222` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:51.585337 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d99e7b-c03c-4a8b-924e-e88a2212dcb6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=26410922fe5847ddb10de90af2b5f3d3, user_id=68edfbc789a648d7b0d604843f322b15, role_id=a2e551773cbd4df084020676865b8222)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:51.588440 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d99e7b-c03c-4a8b-924e-e88a2212dcb6 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:51.593353 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-84c904f4-7c03-4ce5-9c60-a695f2753146 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6fdcfb22d5be47dc9215d4cb0671efb4'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:51.607041 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1009/1996] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 15:09:51 2022] PUT /identity/v3/projects/26410922fe5847ddb10de90af2b5f3d3/users/68edfbc789a648d7b0d604843f322b15/roles/a2e551773cbd4df084020676865b8222 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:09:51.607733 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d99e7b-c03c-4a8b-924e-e88a2212dcb6 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:51.609502 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-b8d99e7b-c03c-4a8b-924e-e88a2212dcb6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:09:51.624028 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-84c904f4-7c03-4ce5-9c60-a695f2753146 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6fdcfb22d5be47dc9215d4cb0671efb4'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:51.627172 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-84c904f4-7c03-4ce5-9c60-a695f2753146 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6fdcfb22d5be47dc9215d4cb0671efb4'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:51.662581 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:DELETE=6,keystone:SELECT=92,keystone:INSERT=2 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:09:51.669271 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-84c904f4-7c03-4ce5-9c60-a695f2753146 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6fdcfb22d5be47dc9215d4cb0671efb4'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:51.672354 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:INSERT=5,keystone:SELECT=85,keystone:DELETE=10 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:09:51.679579 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-84c904f4-7c03-4ce5-9c60-a695f2753146 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6fdcfb22d5be47dc9215d4cb0671efb4'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:51.691975 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 988/1997] 10.222.0.46 () {58 vars in 1223 bytes} [Thu May 26 15:09:51 2022] DELETE /identity/v3/users/6fdcfb22d5be47dc9215d4cb0671efb4 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:09:51.694028 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c904f4-7c03-4ce5-9c60-a695f2753146 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:51.709420 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-84c904f4-7c03-4ce5-9c60-a695f2753146 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:09:51.713891 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d99e7b-c03c-4a8b-924e-e88a2212dcb6 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:51.825746 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2414d0-14e1-4ab2-825d-06442f31bf49 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0156a2f6f3794f479e76b85bdbbe5c26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:51.826631 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2414d0-14e1-4ab2-825d-06442f31bf49 service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:51.827231 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2414d0-14e1-4ab2-825d-06442f31bf49 service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:51.827617 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2414d0-14e1-4ab2-825d-06442f31bf49 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:51.831594 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-ba2414d0-14e1-4ab2-825d-06442f31bf49 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:09:51.870203 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c904f4-7c03-4ce5-9c60-a695f2753146 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:51.879037 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed33bbc-8fbd-4348-a32a-2803c76d4bb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:51.880036 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed33bbc-8fbd-4348-a32a-2803c76d4bb4 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:51.880166 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed33bbc-8fbd-4348-a32a-2803c76d4bb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:51.880407 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed33bbc-8fbd-4348-a32a-2803c76d4bb4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:51.881928 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed33bbc-8fbd-4348-a32a-2803c76d4bb4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:51.883880 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed33bbc-8fbd-4348-a32a-2803c76d4bb4 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:51.897951 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 989/1998] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 15:09:51 2022] GET /identity/v3/roles => generated 1420 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 15:09:51.903615 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed33bbc-8fbd-4348-a32a-2803c76d4bb4 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:51.915180 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed33bbc-8fbd-4348-a32a-2803c76d4bb4 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:51.924189 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f2fb69-52d6-482b-a4bb-6aa592ab45f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:51.925482 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f2fb69-52d6-482b-a4bb-6aa592ab45f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:51.926015 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f2fb69-52d6-482b-a4bb-6aa592ab45f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:51.927184 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f2fb69-52d6-482b-a4bb-6aa592ab45f4 admin admin] PATH_INFO: `/v3/projects/26410922fe5847ddb10de90af2b5f3d3/users/68edfbc789a648d7b0d604843f322b15/roles/a2e551773cbd4df084020676865b8222` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:51.930551 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f2fb69-52d6-482b-a4bb-6aa592ab45f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=26410922fe5847ddb10de90af2b5f3d3, user_id=68edfbc789a648d7b0d604843f322b15, role_id=a2e551773cbd4df084020676865b8222)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:51.938837 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2414d0-14e1-4ab2-825d-06442f31bf49 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:51.939238 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f2fb69-52d6-482b-a4bb-6aa592ab45f4 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:51.940876 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2414d0-14e1-4ab2-825d-06442f31bf49 service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:51.956612 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1010/1999] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:09:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 15:09:51.958351 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 990/2000] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 15:09:51 2022] PUT /identity/v3/projects/26410922fe5847ddb10de90af2b5f3d3/users/68edfbc789a648d7b0d604843f322b15/roles/a2e551773cbd4df084020676865b8222 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:09:51.971055 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa341dd5-99e9-4994-bad4-f52ba91c73eb None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:51.971387 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa341dd5-99e9-4994-bad4-f52ba91c73eb None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:51.971638 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa341dd5-99e9-4994-bad4-f52ba91c73eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:52.010894 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-aa341dd5-99e9-4994-bad4-f52ba91c73eb None None] MFA Rules not processed for user `68edfbc789a648d7b0d604843f322b15`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:09:52.013332 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-aa341dd5-99e9-4994-bad4-f52ba91c73eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:09:52.156867 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1011/2001] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:09:51 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:09:52.168708 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f2fb69-52d6-482b-a4bb-6aa592ab45f4 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:52.171309 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-f8f2fb69-52d6-482b-a4bb-6aa592ab45f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:09:52.220459 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa341dd5-99e9-4994-bad4-f52ba91c73eb None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:52.221960 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-aa341dd5-99e9-4994-bad4-f52ba91c73eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:09:52.257585 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:09:52.326423 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f2fb69-52d6-482b-a4bb-6aa592ab45f4 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:52.332950 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60df697b-a26a-4049-85e6-096a188b0e15 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5b5b35614a547d783e6e77ddb8c1c03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:52.334110 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60df697b-a26a-4049-85e6-096a188b0e15 service cinder] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:52.334483 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60df697b-a26a-4049-85e6-096a188b0e15 service cinder] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:52.334843 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60df697b-a26a-4049-85e6-096a188b0e15 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:52.346075 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa341dd5-99e9-4994-bad4-f52ba91c73eb None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:52.351313 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60df697b-a26a-4049-85e6-096a188b0e15 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:52.353296 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60df697b-a26a-4049-85e6-096a188b0e15 service cinder] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:52.355452 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f9980b-b031-4a55-8e23-4af5eb9e69ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:52.357317 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f9980b-b031-4a55-8e23-4af5eb9e69ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:52.357708 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f9980b-b031-4a55-8e23-4af5eb9e69ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:52.358038 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f9980b-b031-4a55-8e23-4af5eb9e69ee admin admin] PATH_INFO: `/v3/projects/b38f8fdcee1b49be9fcdf84534dc9b46` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:52.359638 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f9980b-b031-4a55-8e23-4af5eb9e69ee admin admin] RBAC: Authorizing `identity:delete_project(project_id=b38f8fdcee1b49be9fcdf84534dc9b46)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:52.362289 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f9980b-b031-4a55-8e23-4af5eb9e69ee admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:52.367219 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 991/2002] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:09:52 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 15:09:52.508360 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60df697b-a26a-4049-85e6-096a188b0e15 service cinder] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:52.510670 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-10f9980b-b031-4a55-8e23-4af5eb9e69ee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b38f8fdcee1b49be9fcdf84534dc9b46', 'request_id': 'req-10f9980b-b031-4a55-8e23-4af5eb9e69ee'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:52.525805 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-10f9980b-b031-4a55-8e23-4af5eb9e69ee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b38f8fdcee1b49be9fcdf84534dc9b46', 'request_id': 'req-10f9980b-b031-4a55-8e23-4af5eb9e69ee'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:52.532059 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60df697b-a26a-4049-85e6-096a188b0e15 service cinder] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:52.537548 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-10f9980b-b031-4a55-8e23-4af5eb9e69ee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b38f8fdcee1b49be9fcdf84534dc9b46', 'request_id': 'req-10f9980b-b031-4a55-8e23-4af5eb9e69ee'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:52.554049 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-10f9980b-b031-4a55-8e23-4af5eb9e69ee admin admin] The token cache is being invalidate because project b38f8fdcee1b49be9fcdf84534dc9b46 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 15:09:52.554467 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-10f9980b-b031-4a55-8e23-4af5eb9e69ee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:52.555090 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-10f9980b-b031-4a55-8e23-4af5eb9e69ee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:52.560255 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1012/2003] 10.222.0.46 () {58 vars in 1232 bytes} [Thu May 26 15:09:52 2022] DELETE /identity/v3/projects/b38f8fdcee1b49be9fcdf84534dc9b46 => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 15:09:52.561367 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f9980b-b031-4a55-8e23-4af5eb9e69ee admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:52.563491 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-10f9980b-b031-4a55-8e23-4af5eb9e69ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:09:52.667496 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f9980b-b031-4a55-8e23-4af5eb9e69ee admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:52.692145 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea355f3-32c3-42e8-9b5f-2ed5b5d44efe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:52.695441 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea355f3-32c3-42e8-9b5f-2ed5b5d44efe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:52.696471 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea355f3-32c3-42e8-9b5f-2ed5b5d44efe admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:52.696770 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea355f3-32c3-42e8-9b5f-2ed5b5d44efe admin admin] PATH_INFO: `/v3/users/69016d2ad4db4bb78570b588a0a8dab0` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:52.700344 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea355f3-32c3-42e8-9b5f-2ed5b5d44efe admin admin] RBAC: Authorizing `identity:delete_user(user_id=69016d2ad4db4bb78570b588a0a8dab0)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:52.706746 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea355f3-32c3-42e8-9b5f-2ed5b5d44efe admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:52.736234 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29685fe1-368c-4c12-9be6-dd33a133c644 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5b5b35614a547d783e6e77ddb8c1c03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:52.764345 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29685fe1-368c-4c12-9be6-dd33a133c644 service cinder] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:52.769597 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29685fe1-368c-4c12-9be6-dd33a133c644 service cinder] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:52.770478 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29685fe1-368c-4c12-9be6-dd33a133c644 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:52.784461 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-29685fe1-368c-4c12-9be6-dd33a133c644 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:09:52.828980 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-3ea355f3-32c3-42e8-9b5f-2ed5b5d44efe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '69016d2ad4db4bb78570b588a0a8dab0'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:52.832120 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:09:52.841839 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-3ea355f3-32c3-42e8-9b5f-2ed5b5d44efe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '69016d2ad4db4bb78570b588a0a8dab0'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:52.842976 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-3ea355f3-32c3-42e8-9b5f-2ed5b5d44efe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '69016d2ad4db4bb78570b588a0a8dab0'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:52.886582 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-3ea355f3-32c3-42e8-9b5f-2ed5b5d44efe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '69016d2ad4db4bb78570b588a0a8dab0'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:52.900818 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:09:52.910545 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-3ea355f3-32c3-42e8-9b5f-2ed5b5d44efe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '69016d2ad4db4bb78570b588a0a8dab0'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:52.918820 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1013/2004] 10.222.0.46 () {58 vars in 1223 bytes} [Thu May 26 15:09:52 2022] DELETE /identity/v3/users/69016d2ad4db4bb78570b588a0a8dab0 => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:09:52.954108 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29685fe1-368c-4c12-9be6-dd33a133c644 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:52.960128 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29685fe1-368c-4c12-9be6-dd33a133c644 service cinder] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:52.974107 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 992/2005] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:09:52 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 466 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 15:09:53.899999 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea355f3-32c3-42e8-9b5f-2ed5b5d44efe admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:53.901716 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-3ea355f3-32c3-42e8-9b5f-2ed5b5d44efe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:09:54.115431 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea355f3-32c3-42e8-9b5f-2ed5b5d44efe admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:54.126161 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87460ba-e0ef-4b20-a94b-27132aa9e0c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:54.129901 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87460ba-e0ef-4b20-a94b-27132aa9e0c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:54.129901 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87460ba-e0ef-4b20-a94b-27132aa9e0c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:54.129901 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87460ba-e0ef-4b20-a94b-27132aa9e0c0 admin admin] PATH_INFO: `/v3/projects/92f36451fefc449cbea3b78dfa0c7c26` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:54.132036 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87460ba-e0ef-4b20-a94b-27132aa9e0c0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=92f36451fefc449cbea3b78dfa0c7c26)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:54.138053 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87460ba-e0ef-4b20-a94b-27132aa9e0c0 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:54.138301 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdf3197-d34f-4451-8a6f-0e01f65de7f3 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:54.138960 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdf3197-d34f-4451-8a6f-0e01f65de7f3 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:54.138960 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffdf3197-d34f-4451-8a6f-0e01f65de7f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:54.187398 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-ffdf3197-d34f-4451-8a6f-0e01f65de7f3 None None] MFA Rules not processed for user `ec18059e0f2943769ae56fbf7a94be6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:09:54.188637 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-ffdf3197-d34f-4451-8a6f-0e01f65de7f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:09:54.221717 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-d87460ba-e0ef-4b20-a94b-27132aa9e0c0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '92f36451fefc449cbea3b78dfa0c7c26', 'request_id': 'req-d87460ba-e0ef-4b20-a94b-27132aa9e0c0'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:54.234550 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-d87460ba-e0ef-4b20-a94b-27132aa9e0c0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '92f36451fefc449cbea3b78dfa0c7c26', 'request_id': 'req-d87460ba-e0ef-4b20-a94b-27132aa9e0c0'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:54.260587 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-d87460ba-e0ef-4b20-a94b-27132aa9e0c0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '92f36451fefc449cbea3b78dfa0c7c26', 'request_id': 'req-d87460ba-e0ef-4b20-a94b-27132aa9e0c0'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:54.288758 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-d87460ba-e0ef-4b20-a94b-27132aa9e0c0 admin admin] The token cache is being invalidate because project 92f36451fefc449cbea3b78dfa0c7c26 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 15:09:54.289117 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-d87460ba-e0ef-4b20-a94b-27132aa9e0c0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:54.290774 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-d87460ba-e0ef-4b20-a94b-27132aa9e0c0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:54.293023 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1014/2006] 10.222.0.46 () {58 vars in 1232 bytes} [Thu May 26 15:09:53 2022] DELETE /identity/v3/projects/92f36451fefc449cbea3b78dfa0c7c26 => generated 0 bytes in 393 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:09:54.298640 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e013357e-b409-4247-b0ad-704d12fed495 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:54.298940 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e013357e-b409-4247-b0ad-704d12fed495 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:54.299226 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e013357e-b409-4247-b0ad-704d12fed495 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:54.315854 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 993/2007] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:09:54 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 15:09:54.322692 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdf3197-d34f-4451-8a6f-0e01f65de7f3 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:54.324886 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-ffdf3197-d34f-4451-8a6f-0e01f65de7f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:09:54.331896 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-e013357e-b409-4247-b0ad-704d12fed495 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:09:54.333576 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-e013357e-b409-4247-b0ad-704d12fed495 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:09:54.342647 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffdf3197-d34f-4451-8a6f-0e01f65de7f3 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:54.397530 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:09:54.416819 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4b1b48-f47d-48ba-8586-20388f171eaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:54.417555 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4b1b48-f47d-48ba-8586-20388f171eaa admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:54.417666 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1015/2008] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:09:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 15:09:54.418640 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4b1b48-f47d-48ba-8586-20388f171eaa admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:54.418926 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4b1b48-f47d-48ba-8586-20388f171eaa admin admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:54.419665 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4b1b48-f47d-48ba-8586-20388f171eaa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:54.421685 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4b1b48-f47d-48ba-8586-20388f171eaa admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:54.423991 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d957b23a-fd34-4b58-b554-b80b8f1ca716 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:54.424204 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d957b23a-fd34-4b58-b554-b80b8f1ca716 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:54.424316 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d957b23a-fd34-4b58-b554-b80b8f1ca716 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:54.446884 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-d957b23a-fd34-4b58-b554-b80b8f1ca716 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:09:54.448176 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-d957b23a-fd34-4b58-b554-b80b8f1ca716 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:09:54.459743 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 994/2009] 10.222.0.46 () {60 vars in 1152 bytes} [Thu May 26 15:09:54 2022] POST /identity/v3/projects => generated 371 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 15:09:54.461924 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1016/2010] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:09:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 15:09:54.463556 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4b1b48-f47d-48ba-8586-20388f171eaa admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:54.466121 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-7f4b1b48-f47d-48ba-8586-20388f171eaa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:09:54.466497 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d957b23a-fd34-4b58-b554-b80b8f1ca716 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:54.478668 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d957b23a-fd34-4b58-b554-b80b8f1ca716 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:54.522699 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:09:54.540341 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4b1b48-f47d-48ba-8586-20388f171eaa admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:54.545108 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66612c71-c302-4d12-9f8a-29235989d7e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:54.545851 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66612c71-c302-4d12-9f8a-29235989d7e3 admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:54.546136 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66612c71-c302-4d12-9f8a-29235989d7e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:54.546550 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66612c71-c302-4d12-9f8a-29235989d7e3 admin admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:54.547573 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66612c71-c302-4d12-9f8a-29235989d7e3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:54.549689 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66612c71-c302-4d12-9f8a-29235989d7e3 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:54.556210 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1703b55a-9d4b-4c4d-b881-75b09126828a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:54.557075 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1703b55a-9d4b-4c4d-b881-75b09126828a admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:54.557364 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1703b55a-9d4b-4c4d-b881-75b09126828a admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:54.557780 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1703b55a-9d4b-4c4d-b881-75b09126828a admin admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:54.559009 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1703b55a-9d4b-4c4d-b881-75b09126828a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 15:09:54.559245 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1703b55a-9d4b-4c4d-b881-75b09126828a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:54.561407 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1703b55a-9d4b-4c4d-b881-75b09126828a admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:54.576022 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 995/2011] 10.222.0.46 () {60 vars in 1143 bytes} [Thu May 26 15:09:54 2022] POST /identity/v3/users => generated 292 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 15:09:54.581289 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1017/2012] 10.222.0.46 () {58 vars in 1152 bytes} [Thu May 26 15:09:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 15:09:54.581487 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66612c71-c302-4d12-9f8a-29235989d7e3 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:54.584016 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1703b55a-9d4b-4c4d-b881-75b09126828a admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:54.598133 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66612c71-c302-4d12-9f8a-29235989d7e3 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:54.606834 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1703b55a-9d4b-4c4d-b881-75b09126828a admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:54.607037 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05e8965-d6db-46c2-a843-e2fa8fe21450 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:54.608088 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05e8965-d6db-46c2-a843-e2fa8fe21450 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:54.608748 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05e8965-d6db-46c2-a843-e2fa8fe21450 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:54.609441 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05e8965-d6db-46c2-a843-e2fa8fe21450 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:54.611419 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05e8965-d6db-46c2-a843-e2fa8fe21450 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:54.614881 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05e8965-d6db-46c2-a843-e2fa8fe21450 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:54.618882 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcafa16-5ebc-46de-9b12-a28f0b5c4742 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:54.624114 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcafa16-5ebc-46de-9b12-a28f0b5c4742 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:54.624654 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcafa16-5ebc-46de-9b12-a28f0b5c4742 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:54.625003 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcafa16-5ebc-46de-9b12-a28f0b5c4742 admin admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:54.626055 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfcafa16-5ebc-46de-9b12-a28f0b5c4742 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:54.628280 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfcafa16-5ebc-46de-9b12-a28f0b5c4742 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:54.637508 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 996/2013] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 15:09:54 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 15:09:54.638899 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05e8965-d6db-46c2-a843-e2fa8fe21450 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:54.650155 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05e8965-d6db-46c2-a843-e2fa8fe21450 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:54.657988 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909bc5d8-441a-439a-800f-7902c4ed323d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:54.658983 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909bc5d8-441a-439a-800f-7902c4ed323d admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:54.659170 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909bc5d8-441a-439a-800f-7902c4ed323d admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:54.659453 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909bc5d8-441a-439a-800f-7902c4ed323d admin admin] PATH_INFO: `/v3/projects/4b9f0b7e0b90413eaf37e607cac8fd54/users/758e9872b34e4aa1aa64c3773ec62b68/roles/a2e551773cbd4df084020676865b8222` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:54.664220 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1018/2014] 10.222.0.46 () {60 vars in 1152 bytes} [Thu May 26 15:09:54 2022] POST /identity/v3/projects => generated 373 bytes in 82 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 15:09:54.667908 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcafa16-5ebc-46de-9b12-a28f0b5c4742 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:54.685215 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909bc5d8-441a-439a-800f-7902c4ed323d admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b9f0b7e0b90413eaf37e607cac8fd54, user_id=758e9872b34e4aa1aa64c3773ec62b68, role_id=a2e551773cbd4df084020676865b8222)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:54.688325 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909bc5d8-441a-439a-800f-7902c4ed323d admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:54.708471 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 997/2015] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 15:09:54 2022] PUT /identity/v3/projects/4b9f0b7e0b90413eaf37e607cac8fd54/users/758e9872b34e4aa1aa64c3773ec62b68/roles/a2e551773cbd4df084020676865b8222 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 15:09:54.710136 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909bc5d8-441a-439a-800f-7902c4ed323d admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:54.755351 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcafa16-5ebc-46de-9b12-a28f0b5c4742 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:54.796375 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909bc5d8-441a-439a-800f-7902c4ed323d admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:54.802348 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad36edf3-a91a-4ebd-a4e7-c4b67dd4c777 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:54.803193 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad36edf3-a91a-4ebd-a4e7-c4b67dd4c777 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:54.803789 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad36edf3-a91a-4ebd-a4e7-c4b67dd4c777 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:54.803789 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad36edf3-a91a-4ebd-a4e7-c4b67dd4c777 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:54.804494 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad36edf3-a91a-4ebd-a4e7-c4b67dd4c777 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:54.806320 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad36edf3-a91a-4ebd-a4e7-c4b67dd4c777 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:54.816891 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 998/2016] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 15:09:54 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 15:09:54.821173 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad36edf3-a91a-4ebd-a4e7-c4b67dd4c777 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:54.829306 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d305ce7-cf5c-4e22-8eb9-6375b1e36156 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:54.830218 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d305ce7-cf5c-4e22-8eb9-6375b1e36156 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:54.830572 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d305ce7-cf5c-4e22-8eb9-6375b1e36156 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:54.830923 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d305ce7-cf5c-4e22-8eb9-6375b1e36156 admin admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:54.831959 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d305ce7-cf5c-4e22-8eb9-6375b1e36156 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:54.834277 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d305ce7-cf5c-4e22-8eb9-6375b1e36156 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:54.834492 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad36edf3-a91a-4ebd-a4e7-c4b67dd4c777 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:54.841012 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff94379d-4b01-4cb5-9a4f-88f1d61ca5e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:54.842153 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff94379d-4b01-4cb5-9a4f-88f1d61ca5e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:54.842811 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff94379d-4b01-4cb5-9a4f-88f1d61ca5e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:54.843095 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff94379d-4b01-4cb5-9a4f-88f1d61ca5e0 admin admin] PATH_INFO: `/v3/projects/4b9f0b7e0b90413eaf37e607cac8fd54/users/758e9872b34e4aa1aa64c3773ec62b68/roles/a2e551773cbd4df084020676865b8222` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:54.848622 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff94379d-4b01-4cb5-9a4f-88f1d61ca5e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b9f0b7e0b90413eaf37e607cac8fd54, user_id=758e9872b34e4aa1aa64c3773ec62b68, role_id=a2e551773cbd4df084020676865b8222)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:54.852249 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff94379d-4b01-4cb5-9a4f-88f1d61ca5e0 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:54.857839 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:09:54.863969 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1019/2017] 10.222.0.46 () {60 vars in 1143 bytes} [Thu May 26 15:09:54 2022] POST /identity/v3/users => generated 287 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 15:09:54.867366 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d305ce7-cf5c-4e22-8eb9-6375b1e36156 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:54.867940 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 999/2018] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 15:09:54 2022] PUT /identity/v3/projects/4b9f0b7e0b90413eaf37e607cac8fd54/users/758e9872b34e4aa1aa64c3773ec62b68/roles/a2e551773cbd4df084020676865b8222 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 15:09:54.883676 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d305ce7-cf5c-4e22-8eb9-6375b1e36156 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:54.983909 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8e6ec0-5ae3-49bf-8bd2-1bddb26c3f82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:54.985120 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8e6ec0-5ae3-49bf-8bd2-1bddb26c3f82 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:54.985120 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8e6ec0-5ae3-49bf-8bd2-1bddb26c3f82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:54.985555 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8e6ec0-5ae3-49bf-8bd2-1bddb26c3f82 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:54.986373 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8e6ec0-5ae3-49bf-8bd2-1bddb26c3f82 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:54.988236 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8e6ec0-5ae3-49bf-8bd2-1bddb26c3f82 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:55.000603 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1020/2019] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 15:09:54 2022] GET /identity/v3/roles => generated 1420 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 15:09:55.006421 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff94379d-4b01-4cb5-9a4f-88f1d61ca5e0 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:55.017921 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff94379d-4b01-4cb5-9a4f-88f1d61ca5e0 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:55.025075 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3199d7f7-9e3c-42d2-8f8a-b01598c2218e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:55.025809 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3199d7f7-9e3c-42d2-8f8a-b01598c2218e admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:55.026092 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3199d7f7-9e3c-42d2-8f8a-b01598c2218e admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:55.026378 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3199d7f7-9e3c-42d2-8f8a-b01598c2218e admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:55.027200 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3199d7f7-9e3c-42d2-8f8a-b01598c2218e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:55.028936 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3199d7f7-9e3c-42d2-8f8a-b01598c2218e admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:55.046149 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1000/2020] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 15:09:55 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 15:09:55.048987 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8e6ec0-5ae3-49bf-8bd2-1bddb26c3f82 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:55.065382 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8e6ec0-5ae3-49bf-8bd2-1bddb26c3f82 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:55.075766 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9e19dd-111b-47be-a5cd-b30fed31238e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:55.076747 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9e19dd-111b-47be-a5cd-b30fed31238e admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:55.077013 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9e19dd-111b-47be-a5cd-b30fed31238e admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:55.077333 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9e19dd-111b-47be-a5cd-b30fed31238e admin admin] PATH_INFO: `/v3/projects/664448e66c54440f909ae6727ccc3deb/users/1c54e915f4014d58ba28d6e6d5dd6ab6/roles/d4088d1845a24fbd9bd8975ce0b1d3d0` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:55.101644 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9e19dd-111b-47be-a5cd-b30fed31238e admin admin] RBAC: Authorizing `identity:create_grant(project_id=664448e66c54440f909ae6727ccc3deb, user_id=1c54e915f4014d58ba28d6e6d5dd6ab6, role_id=d4088d1845a24fbd9bd8975ce0b1d3d0)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:55.107065 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9e19dd-111b-47be-a5cd-b30fed31238e admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:55.132562 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1021/2021] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 15:09:55 2022] PUT /identity/v3/projects/664448e66c54440f909ae6727ccc3deb/users/1c54e915f4014d58ba28d6e6d5dd6ab6/roles/d4088d1845a24fbd9bd8975ce0b1d3d0 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:09:55.138747 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5c427a-dbc8-4556-a1b5-3d531c7d781c None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:55.139231 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5c427a-dbc8-4556-a1b5-3d531c7d781c None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:55.139512 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5c427a-dbc8-4556-a1b5-3d531c7d781c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:55.157613 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-bc5c427a-dbc8-4556-a1b5-3d531c7d781c None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:09:55.158919 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-bc5c427a-dbc8-4556-a1b5-3d531c7d781c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:09:55.229948 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1001/2022] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:09:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:09:55.241175 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9e19dd-111b-47be-a5cd-b30fed31238e admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:55.243031 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-3d9e19dd-111b-47be-a5cd-b30fed31238e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:09:55.321594 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9e19dd-111b-47be-a5cd-b30fed31238e admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:55.327435 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fb833b-b000-4d04-a521-07cc59eeb6a3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0156a2f6f3794f479e76b85bdbbe5c26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:55.328729 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fb833b-b000-4d04-a521-07cc59eeb6a3 service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:55.329023 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fb833b-b000-4d04-a521-07cc59eeb6a3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:55.329310 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fb833b-b000-4d04-a521-07cc59eeb6a3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:55.344062 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fb833b-b000-4d04-a521-07cc59eeb6a3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:55.346093 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fb833b-b000-4d04-a521-07cc59eeb6a3 service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:55.359243 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1022/2023] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:09:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 15:09:55.994324 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f985e60d-0849-4b6a-9cbe-f3f321dc9924 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:55.994615 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f985e60d-0849-4b6a-9cbe-f3f321dc9924 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:55.994892 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f985e60d-0849-4b6a-9cbe-f3f321dc9924 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:56.025301 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:09:56.030350 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-f985e60d-0849-4b6a-9cbe-f3f321dc9924 None None] MFA Rules not processed for user `758e9872b34e4aa1aa64c3773ec62b68`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:09:56.033230 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-f985e60d-0849-4b6a-9cbe-f3f321dc9924 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:09:56.181279 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1002/2024] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:09:55 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:09:56.190584 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fb833b-b000-4d04-a521-07cc59eeb6a3 service neutron] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:56.193538 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-97fb833b-b000-4d04-a521-07cc59eeb6a3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:09:56.284005 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fb833b-b000-4d04-a521-07cc59eeb6a3 service neutron] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:56.290263 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5831b8f2-c3bd-45df-b987-ffc4cc0ddcd9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5b5b35614a547d783e6e77ddb8c1c03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:56.291254 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5831b8f2-c3bd-45df-b987-ffc4cc0ddcd9 service cinder] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:56.291778 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5831b8f2-c3bd-45df-b987-ffc4cc0ddcd9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:56.292428 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5831b8f2-c3bd-45df-b987-ffc4cc0ddcd9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:56.296373 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-5831b8f2-c3bd-45df-b987-ffc4cc0ddcd9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:09:56.366155 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:09:56.408314 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5831b8f2-c3bd-45df-b987-ffc4cc0ddcd9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:56.409818 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5831b8f2-c3bd-45df-b987-ffc4cc0ddcd9 service cinder] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:56.420014 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1023/2025] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:09:56 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 15:09:56.535055 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58591105-952e-4e00-8730-4883eabab7b2 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:56.536142 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58591105-952e-4e00-8730-4883eabab7b2 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:56.536528 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58591105-952e-4e00-8730-4883eabab7b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:56.570615 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-58591105-952e-4e00-8730-4883eabab7b2 None None] MFA Rules not processed for user `1c54e915f4014d58ba28d6e6d5dd6ab6`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:09:56.572191 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-58591105-952e-4e00-8730-4883eabab7b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:09:56.674862 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1003/2026] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:09:56 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:09:56.689528 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5831b8f2-c3bd-45df-b987-ffc4cc0ddcd9 service cinder] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:56.691005 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-5831b8f2-c3bd-45df-b987-ffc4cc0ddcd9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:09:56.706278 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5831b8f2-c3bd-45df-b987-ffc4cc0ddcd9 service cinder] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:56.710457 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affb508b-77d0-4f09-be5e-d33ad14e9772 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5b5b35614a547d783e6e77ddb8c1c03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:56.711219 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affb508b-77d0-4f09-be5e-d33ad14e9772 service cinder] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:56.711502 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affb508b-77d0-4f09-be5e-d33ad14e9772 service cinder] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:56.711777 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-affb508b-77d0-4f09-be5e-d33ad14e9772 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:56.723453 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-affb508b-77d0-4f09-be5e-d33ad14e9772 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:56.736961 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-affb508b-77d0-4f09-be5e-d33ad14e9772 service cinder] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:56.746837 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1024/2027] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:09:56 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 15:09:57.857888 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58591105-952e-4e00-8730-4883eabab7b2 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:57.867787 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58591105-952e-4e00-8730-4883eabab7b2 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:57.872230 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b57619-d573-4e9c-913d-696109f2394a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5b5b35614a547d783e6e77ddb8c1c03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:57.872965 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b57619-d573-4e9c-913d-696109f2394a service cinder] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:57.873556 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b57619-d573-4e9c-913d-696109f2394a service cinder] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:57.873843 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b57619-d573-4e9c-913d-696109f2394a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:57.883692 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b57619-d573-4e9c-913d-696109f2394a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:57.885053 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b57619-d573-4e9c-913d-696109f2394a service cinder] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:57.894995 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1004/2028] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:09:57 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 15:09:58.645712 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affb508b-77d0-4f09-be5e-d33ad14e9772 service cinder] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:58.647901 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-affb508b-77d0-4f09-be5e-d33ad14e9772 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:09:58.664377 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-affb508b-77d0-4f09-be5e-d33ad14e9772 service cinder] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:58.669733 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ac4707-6131-4476-ad23-39fe52f0151f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:58.670618 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ac4707-6131-4476-ad23-39fe52f0151f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:58.670947 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ac4707-6131-4476-ad23-39fe52f0151f admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:58.671327 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ac4707-6131-4476-ad23-39fe52f0151f admin admin] PATH_INFO: `/v3/users/0425558227fb4feeba1898602aeefba6` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:58.673037 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ac4707-6131-4476-ad23-39fe52f0151f admin admin] RBAC: Authorizing `identity:delete_user(user_id=0425558227fb4feeba1898602aeefba6)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:58.674753 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ac4707-6131-4476-ad23-39fe52f0151f admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:58.731080 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-a7ac4707-6131-4476-ad23-39fe52f0151f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0425558227fb4feeba1898602aeefba6'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:58.746357 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-a7ac4707-6131-4476-ad23-39fe52f0151f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0425558227fb4feeba1898602aeefba6'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:58.747507 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-a7ac4707-6131-4476-ad23-39fe52f0151f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0425558227fb4feeba1898602aeefba6'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:58.768354 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-a7ac4707-6131-4476-ad23-39fe52f0151f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0425558227fb4feeba1898602aeefba6'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:58.775234 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-a7ac4707-6131-4476-ad23-39fe52f0151f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0425558227fb4feeba1898602aeefba6'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:58.782165 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1025/2029] 10.222.0.46 () {58 vars in 1223 bytes} [Thu May 26 15:09:58 2022] DELETE /identity/v3/users/0425558227fb4feeba1898602aeefba6 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:09:58.786872 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f8d7f7-c93e-484e-95ad-b0e25e62a1ea None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:58.789870 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f8d7f7-c93e-484e-95ad-b0e25e62a1ea None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:58.790181 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f8d7f7-c93e-484e-95ad-b0e25e62a1ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:58.808746 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-67f8d7f7-c93e-484e-95ad-b0e25e62a1ea None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:09:58.810217 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-67f8d7f7-c93e-484e-95ad-b0e25e62a1ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:09:58.873918 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1005/2030] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:09:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:09:58.884661 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ac4707-6131-4476-ad23-39fe52f0151f admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:58.886156 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-a7ac4707-6131-4476-ad23-39fe52f0151f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:09:58.945189 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ac4707-6131-4476-ad23-39fe52f0151f admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:58.950297 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bd841c-8686-48f4-9246-9308c479aa49 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0156a2f6f3794f479e76b85bdbbe5c26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:58.950655 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bd841c-8686-48f4-9246-9308c479aa49 service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:58.950933 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bd841c-8686-48f4-9246-9308c479aa49 service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:58.951222 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79bd841c-8686-48f4-9246-9308c479aa49 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:58.961514 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79bd841c-8686-48f4-9246-9308c479aa49 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:58.963509 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79bd841c-8686-48f4-9246-9308c479aa49 service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:58.975477 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1026/2031] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:09:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 15:09:59.164556 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd49a56-97b7-4be9-90f0-bdfa394692f0 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:59.164834 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd49a56-97b7-4be9-90f0-bdfa394692f0 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:59.165128 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd49a56-97b7-4be9-90f0-bdfa394692f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:59.190097 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-3bd49a56-97b7-4be9-90f0-bdfa394692f0 None None] MFA Rules not processed for user `1c54e915f4014d58ba28d6e6d5dd6ab6`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:09:59.191559 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-3bd49a56-97b7-4be9-90f0-bdfa394692f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:09:59.203267 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:09:59.267992 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1006/2032] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:09:59 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:09:59.273295 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bd841c-8686-48f4-9246-9308c479aa49 service neutron] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:59.274983 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-79bd841c-8686-48f4-9246-9308c479aa49 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:09:59.301883 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79bd841c-8686-48f4-9246-9308c479aa49 service neutron] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:59.307846 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd56832-f761-44f5-bc2d-e73416ff7a2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:59.309786 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd56832-f761-44f5-bc2d-e73416ff7a2d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:59.309786 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd56832-f761-44f5-bc2d-e73416ff7a2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:59.309786 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd56832-f761-44f5-bc2d-e73416ff7a2d admin admin] PATH_INFO: `/v3/projects/6f563cb6ba894aab9ad9aa10193a6bd6` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:59.311086 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd56832-f761-44f5-bc2d-e73416ff7a2d admin admin] RBAC: Authorizing `identity:delete_project(project_id=6f563cb6ba894aab9ad9aa10193a6bd6)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:59.313330 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd56832-f761-44f5-bc2d-e73416ff7a2d admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:59.354776 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:09:59.383165 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-5cd56832-f761-44f5-bc2d-e73416ff7a2d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6f563cb6ba894aab9ad9aa10193a6bd6', 'request_id': 'req-5cd56832-f761-44f5-bc2d-e73416ff7a2d'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:59.394200 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-5cd56832-f761-44f5-bc2d-e73416ff7a2d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6f563cb6ba894aab9ad9aa10193a6bd6', 'request_id': 'req-5cd56832-f761-44f5-bc2d-e73416ff7a2d'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:59.401223 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-5cd56832-f761-44f5-bc2d-e73416ff7a2d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6f563cb6ba894aab9ad9aa10193a6bd6', 'request_id': 'req-5cd56832-f761-44f5-bc2d-e73416ff7a2d'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:59.414393 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-5cd56832-f761-44f5-bc2d-e73416ff7a2d admin admin] The token cache is being invalidate because project 6f563cb6ba894aab9ad9aa10193a6bd6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 15:09:59.414700 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-5cd56832-f761-44f5-bc2d-e73416ff7a2d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:59.415432 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-5cd56832-f761-44f5-bc2d-e73416ff7a2d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:59.417336 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1027/2033] 10.222.0.46 () {58 vars in 1232 bytes} [Thu May 26 15:09:59 2022] DELETE /identity/v3/projects/6f563cb6ba894aab9ad9aa10193a6bd6 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:09:59.421553 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd49a56-97b7-4be9-90f0-bdfa394692f0 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:09:59.440271 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd49a56-97b7-4be9-90f0-bdfa394692f0 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:09:59.549237 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3145d8ca-dc37-42d2-b853-f8ef0473f87e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:09:59.551037 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3145d8ca-dc37-42d2-b853-f8ef0473f87e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:09:59.551479 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3145d8ca-dc37-42d2-b853-f8ef0473f87e admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:09:59.551864 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3145d8ca-dc37-42d2-b853-f8ef0473f87e admin admin] PATH_INFO: `/v3/users/68edfbc789a648d7b0d604843f322b15` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:09:59.553429 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3145d8ca-dc37-42d2-b853-f8ef0473f87e admin admin] RBAC: Authorizing `identity:delete_user(user_id=68edfbc789a648d7b0d604843f322b15)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:09:59.555754 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3145d8ca-dc37-42d2-b853-f8ef0473f87e admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:09:59.676438 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-3145d8ca-dc37-42d2-b853-f8ef0473f87e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '68edfbc789a648d7b0d604843f322b15'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:59.690498 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-3145d8ca-dc37-42d2-b853-f8ef0473f87e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '68edfbc789a648d7b0d604843f322b15'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:59.691891 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-3145d8ca-dc37-42d2-b853-f8ef0473f87e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '68edfbc789a648d7b0d604843f322b15'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:59.714628 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-3145d8ca-dc37-42d2-b853-f8ef0473f87e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '68edfbc789a648d7b0d604843f322b15'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:59.739884 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-3145d8ca-dc37-42d2-b853-f8ef0473f87e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '68edfbc789a648d7b0d604843f322b15'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:09:59.750586 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1007/2034] 10.222.0.46 () {58 vars in 1223 bytes} [Thu May 26 15:09:59 2022] DELETE /identity/v3/users/68edfbc789a648d7b0d604843f322b15 => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:10:00.137881 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd56832-f761-44f5-bc2d-e73416ff7a2d admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:10:00.139291 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-5cd56832-f761-44f5-bc2d-e73416ff7a2d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:10:00.174725 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd56832-f761-44f5-bc2d-e73416ff7a2d admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:10:00.240111 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ec782c-6426-405b-8d68-667e41cadf56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:10:00.240997 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ec782c-6426-405b-8d68-667e41cadf56 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:10:00.241277 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ec782c-6426-405b-8d68-667e41cadf56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:10:00.241586 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ec782c-6426-405b-8d68-667e41cadf56 admin admin] PATH_INFO: `/v3/projects/26410922fe5847ddb10de90af2b5f3d3` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:10:00.242905 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ec782c-6426-405b-8d68-667e41cadf56 admin admin] RBAC: Authorizing `identity:delete_project(project_id=26410922fe5847ddb10de90af2b5f3d3)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:10:00.244805 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ec782c-6426-405b-8d68-667e41cadf56 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:10:00.297152 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-31ec782c-6426-405b-8d68-667e41cadf56 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '26410922fe5847ddb10de90af2b5f3d3', 'request_id': 'req-31ec782c-6426-405b-8d68-667e41cadf56'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:10:00.306699 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-31ec782c-6426-405b-8d68-667e41cadf56 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '26410922fe5847ddb10de90af2b5f3d3', 'request_id': 'req-31ec782c-6426-405b-8d68-667e41cadf56'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:10:00.312966 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-31ec782c-6426-405b-8d68-667e41cadf56 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '26410922fe5847ddb10de90af2b5f3d3', 'request_id': 'req-31ec782c-6426-405b-8d68-667e41cadf56'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:10:00.325150 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-31ec782c-6426-405b-8d68-667e41cadf56 admin admin] The token cache is being invalidate because project 26410922fe5847ddb10de90af2b5f3d3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 15:10:00.325428 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-31ec782c-6426-405b-8d68-667e41cadf56 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:10:00.326104 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-31ec782c-6426-405b-8d68-667e41cadf56 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:10:00.328083 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1028/2035] 10.222.0.46 () {58 vars in 1232 bytes} [Thu May 26 15:10:00 2022] DELETE /identity/v3/projects/26410922fe5847ddb10de90af2b5f3d3 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:10:00.333088 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302684a7-692e-4590-8a09-947282b7777e None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:10:00.333633 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302684a7-692e-4590-8a09-947282b7777e None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:10:00.333880 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302684a7-692e-4590-8a09-947282b7777e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:10:00.354572 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-302684a7-692e-4590-8a09-947282b7777e None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:10:00.356883 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-302684a7-692e-4590-8a09-947282b7777e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:10:00.400475 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:10:00.431367 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1008/2036] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:10:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:10:00.442919 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d42fb0-d660-4a97-aeb5-09bd57ffd9d0 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:10:00.443210 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d42fb0-d660-4a97-aeb5-09bd57ffd9d0 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:10:00.443494 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d42fb0-d660-4a97-aeb5-09bd57ffd9d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:10:00.467083 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-73d42fb0-d660-4a97-aeb5-09bd57ffd9d0 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:10:00.468599 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-73d42fb0-d660-4a97-aeb5-09bd57ffd9d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:10:00.479760 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1029/2037] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:10:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:10:00.480955 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302684a7-692e-4590-8a09-947282b7777e None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:10:00.490756 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302684a7-692e-4590-8a09-947282b7777e None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:10:00.495561 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0b33c9-e1d7-482c-aaec-b4e62c708ec1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:10:00.496353 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0b33c9-e1d7-482c-aaec-b4e62c708ec1 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:10:00.496639 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0b33c9-e1d7-482c-aaec-b4e62c708ec1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:10:00.496958 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0b33c9-e1d7-482c-aaec-b4e62c708ec1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:10:00.497769 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0b33c9-e1d7-482c-aaec-b4e62c708ec1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 15:10:00.497960 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0b33c9-e1d7-482c-aaec-b4e62c708ec1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:10:00.500066 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0b33c9-e1d7-482c-aaec-b4e62c708ec1 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:10:00.513620 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1009/2038] 10.222.0.46 () {58 vars in 1152 bytes} [Thu May 26 15:10:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 15:10:00.517861 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d42fb0-d660-4a97-aeb5-09bd57ffd9d0 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:10:00.528973 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d42fb0-d660-4a97-aeb5-09bd57ffd9d0 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:10:00.534466 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f5955a-8424-438d-a55e-e9e8a2ee29a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:10:00.535129 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f5955a-8424-438d-a55e-e9e8a2ee29a8 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:10:00.535480 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f5955a-8424-438d-a55e-e9e8a2ee29a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:10:00.535734 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f5955a-8424-438d-a55e-e9e8a2ee29a8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:10:00.536459 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f5955a-8424-438d-a55e-e9e8a2ee29a8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:10:00.538782 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f5955a-8424-438d-a55e-e9e8a2ee29a8 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:10:00.576771 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1030/2039] 10.222.0.46 () {60 vars in 1152 bytes} [Thu May 26 15:10:00 2022] POST /identity/v3/projects => generated 385 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 15:10:00.578177 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0b33c9-e1d7-482c-aaec-b4e62c708ec1 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:10:00.588290 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0b33c9-e1d7-482c-aaec-b4e62c708ec1 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:10:00.681060 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9d26cc-dd33-4239-a320-f951b995b214 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:10:00.681940 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9d26cc-dd33-4239-a320-f951b995b214 admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:10:00.682352 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9d26cc-dd33-4239-a320-f951b995b214 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:10:00.682839 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9d26cc-dd33-4239-a320-f951b995b214 admin admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:10:00.683726 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9d26cc-dd33-4239-a320-f951b995b214 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:10:00.686178 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9d26cc-dd33-4239-a320-f951b995b214 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:10:00.725600 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1010/2040] 10.222.0.46 () {60 vars in 1143 bytes} [Thu May 26 15:10:00 2022] POST /identity/v3/users => generated 293 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 15:10:00.728433 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f5955a-8424-438d-a55e-e9e8a2ee29a8 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:10:00.741287 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f5955a-8424-438d-a55e-e9e8a2ee29a8 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:10:00.746556 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a3d654-4f2a-43ef-abb6-61bce704c7eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:10:00.747222 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a3d654-4f2a-43ef-abb6-61bce704c7eb admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:10:00.747508 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a3d654-4f2a-43ef-abb6-61bce704c7eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:10:00.747837 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a3d654-4f2a-43ef-abb6-61bce704c7eb admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:10:00.748835 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a3d654-4f2a-43ef-abb6-61bce704c7eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:10:00.750525 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a3d654-4f2a-43ef-abb6-61bce704c7eb admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:10:00.765264 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1031/2041] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 15:10:00 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 15:10:00.766435 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9d26cc-dd33-4239-a320-f951b995b214 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:10:00.777890 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9d26cc-dd33-4239-a320-f951b995b214 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:10:00.783630 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65c5b5e-36da-4a71-ad2b-b47e52ed6a26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:10:00.785047 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65c5b5e-36da-4a71-ad2b-b47e52ed6a26 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:10:00.785292 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65c5b5e-36da-4a71-ad2b-b47e52ed6a26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:10:00.785717 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65c5b5e-36da-4a71-ad2b-b47e52ed6a26 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:10:00.787003 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65c5b5e-36da-4a71-ad2b-b47e52ed6a26 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:10:00.788335 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65c5b5e-36da-4a71-ad2b-b47e52ed6a26 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:10:00.813845 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a3d654-4f2a-43ef-abb6-61bce704c7eb admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:10:00.814373 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1011/2042] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 15:10:00 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 15:10:00.821055 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:DELETE=13,keystone:SELECT=84,keystone:INSERT=3 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:10:00.828976 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a3d654-4f2a-43ef-abb6-61bce704c7eb admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:10:00.837374 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22332ec-503d-4563-a3e8-8ec2a1823c06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:10:00.838344 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22332ec-503d-4563-a3e8-8ec2a1823c06 admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:10:00.838804 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22332ec-503d-4563-a3e8-8ec2a1823c06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:10:00.839015 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22332ec-503d-4563-a3e8-8ec2a1823c06 admin admin] PATH_INFO: `/v3/projects/4ee70304b69f476b9e200149ecc51efe/users/f89c99ce8e76462cad97ee91aa482f4c/roles/d4088d1845a24fbd9bd8975ce0b1d3d0` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:10:00.857632 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22332ec-503d-4563-a3e8-8ec2a1823c06 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4ee70304b69f476b9e200149ecc51efe, user_id=f89c99ce8e76462cad97ee91aa482f4c, role_id=d4088d1845a24fbd9bd8975ce0b1d3d0)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:10:00.860445 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22332ec-503d-4563-a3e8-8ec2a1823c06 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:10:00.877698 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1032/2043] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 15:10:00 2022] PUT /identity/v3/projects/4ee70304b69f476b9e200149ecc51efe/users/f89c99ce8e76462cad97ee91aa482f4c/roles/d4088d1845a24fbd9bd8975ce0b1d3d0 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 15:10:00.892251 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbb0e1b-05fd-462e-a8a0-317adfab0c1a None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:10:00.892651 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbb0e1b-05fd-462e-a8a0-317adfab0c1a None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:10:00.893063 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbb0e1b-05fd-462e-a8a0-317adfab0c1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:10:00.930749 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-4dbb0e1b-05fd-462e-a8a0-317adfab0c1a None None] MFA Rules not processed for user `f89c99ce8e76462cad97ee91aa482f4c`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:10:00.932052 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-4dbb0e1b-05fd-462e-a8a0-317adfab0c1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:10:01.005775 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1012/2044] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:10:00 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:10:01.005977 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22332ec-503d-4563-a3e8-8ec2a1823c06 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:10:01.069856 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22332ec-503d-4563-a3e8-8ec2a1823c06 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:10:01.074499 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863948d2-be85-4a76-9603-72ff0f3ee0e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:10:01.075152 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863948d2-be85-4a76-9603-72ff0f3ee0e4 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:10:01.075437 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863948d2-be85-4a76-9603-72ff0f3ee0e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:10:01.075711 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863948d2-be85-4a76-9603-72ff0f3ee0e4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:10:01.076397 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863948d2-be85-4a76-9603-72ff0f3ee0e4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:10:01.078634 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863948d2-be85-4a76-9603-72ff0f3ee0e4 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:10:01.111059 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1033/2045] 10.222.0.46 () {60 vars in 1152 bytes} [Thu May 26 15:10:01 2022] POST /identity/v3/projects => generated 385 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 15:10:01.112422 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbb0e1b-05fd-462e-a8a0-317adfab0c1a None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:10:01.130988 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbb0e1b-05fd-462e-a8a0-317adfab0c1a None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:10:01.187066 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:10:01.257964 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6e0427-ffe6-478f-99ef-f1e163c38b1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:10:01.260611 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6e0427-ffe6-478f-99ef-f1e163c38b1f admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:10:01.265746 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6e0427-ffe6-478f-99ef-f1e163c38b1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:10:01.266231 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6e0427-ffe6-478f-99ef-f1e163c38b1f admin admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:10:01.269426 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6e0427-ffe6-478f-99ef-f1e163c38b1f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:10:01.273633 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6e0427-ffe6-478f-99ef-f1e163c38b1f admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:10:01.310439 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1013/2046] 10.222.0.46 () {60 vars in 1143 bytes} [Thu May 26 15:10:01 2022] POST /identity/v3/users => generated 293 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 15:10:01.314597 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863948d2-be85-4a76-9603-72ff0f3ee0e4 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:10:01.326171 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863948d2-be85-4a76-9603-72ff0f3ee0e4 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:10:01.330938 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91362df0-56be-4136-ba20-f871761ae3c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:10:01.331819 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91362df0-56be-4136-ba20-f871761ae3c1 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:10:01.332103 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91362df0-56be-4136-ba20-f871761ae3c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:10:01.332383 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91362df0-56be-4136-ba20-f871761ae3c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:10:01.333114 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91362df0-56be-4136-ba20-f871761ae3c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:10:01.334822 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91362df0-56be-4136-ba20-f871761ae3c1 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:10:01.347414 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1034/2047] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 15:10:01 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 15:10:01.351837 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6e0427-ffe6-478f-99ef-f1e163c38b1f admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:10:01.365274 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6e0427-ffe6-478f-99ef-f1e163c38b1f admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:10:01.380458 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee4d1c4-43a8-45cf-acb8-0062ac171f91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:10:01.381386 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee4d1c4-43a8-45cf-acb8-0062ac171f91 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:10:01.382285 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee4d1c4-43a8-45cf-acb8-0062ac171f91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:10:01.382599 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee4d1c4-43a8-45cf-acb8-0062ac171f91 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:10:01.383344 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee4d1c4-43a8-45cf-acb8-0062ac171f91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:10:01.385024 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee4d1c4-43a8-45cf-acb8-0062ac171f91 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:10:01.396024 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1014/2048] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 15:10:01 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 15:10:01.400312 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91362df0-56be-4136-ba20-f871761ae3c1 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:10:01.410458 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91362df0-56be-4136-ba20-f871761ae3c1 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:10:01.415364 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14285fab-1c14-4ef6-bb32-c5606d22edac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:10:01.416476 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14285fab-1c14-4ef6-bb32-c5606d22edac admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:10:01.416909 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14285fab-1c14-4ef6-bb32-c5606d22edac admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:10:01.417913 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14285fab-1c14-4ef6-bb32-c5606d22edac admin admin] PATH_INFO: `/v3/projects/004344bc46f04195ad9339a593b0b661/users/9740fe51369e4163b8ad0783030b1209/roles/d4088d1845a24fbd9bd8975ce0b1d3d0` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:10:01.446243 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14285fab-1c14-4ef6-bb32-c5606d22edac admin admin] RBAC: Authorizing `identity:create_grant(project_id=004344bc46f04195ad9339a593b0b661, user_id=9740fe51369e4163b8ad0783030b1209, role_id=d4088d1845a24fbd9bd8975ce0b1d3d0)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:10:01.452904 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14285fab-1c14-4ef6-bb32-c5606d22edac admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:10:01.477040 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1035/2049] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 15:10:01 2022] PUT /identity/v3/projects/004344bc46f04195ad9339a593b0b661/users/9740fe51369e4163b8ad0783030b1209/roles/d4088d1845a24fbd9bd8975ce0b1d3d0 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 15:10:01.490546 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa0d8c7-7c16-4396-8388-62e5956ff82c None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:10:01.492030 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa0d8c7-7c16-4396-8388-62e5956ff82c None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:10:01.492326 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa0d8c7-7c16-4396-8388-62e5956ff82c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:10:01.524246 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-0aa0d8c7-7c16-4396-8388-62e5956ff82c None None] MFA Rules not processed for user `9740fe51369e4163b8ad0783030b1209`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:10:01.525503 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-0aa0d8c7-7c16-4396-8388-62e5956ff82c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:10:01.603844 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1015/2050] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:10:01 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:10:01.608844 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14285fab-1c14-4ef6-bb32-c5606d22edac admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:10:01.676417 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14285fab-1c14-4ef6-bb32-c5606d22edac admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:10:01.681726 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164f0eee-487f-4955-80a1-e1f9526083e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:10:01.682395 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164f0eee-487f-4955-80a1-e1f9526083e6 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:10:01.682682 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164f0eee-487f-4955-80a1-e1f9526083e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:10:01.683001 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164f0eee-487f-4955-80a1-e1f9526083e6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:10:01.683700 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164f0eee-487f-4955-80a1-e1f9526083e6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:10:01.686034 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164f0eee-487f-4955-80a1-e1f9526083e6 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:10:01.715099 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1036/2051] 10.222.0.46 () {60 vars in 1152 bytes} [Thu May 26 15:10:01 2022] POST /identity/v3/projects => generated 385 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 15:10:01.718799 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa0d8c7-7c16-4396-8388-62e5956ff82c None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:10:01.729995 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa0d8c7-7c16-4396-8388-62e5956ff82c None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:10:01.787489 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb03953c-9680-4e49-8d6c-c505272babf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:10:01.788173 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb03953c-9680-4e49-8d6c-c505272babf9 admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:10:01.788454 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb03953c-9680-4e49-8d6c-c505272babf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:10:01.788735 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb03953c-9680-4e49-8d6c-c505272babf9 admin admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:10:01.789466 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb03953c-9680-4e49-8d6c-c505272babf9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:10:01.791409 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb03953c-9680-4e49-8d6c-c505272babf9 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:10:01.815121 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1016/2052] 10.222.0.46 () {60 vars in 1143 bytes} [Thu May 26 15:10:01 2022] POST /identity/v3/users => generated 299 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 15:10:01.819476 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164f0eee-487f-4955-80a1-e1f9526083e6 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:10:01.830612 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164f0eee-487f-4955-80a1-e1f9526083e6 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:10:01.836408 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5507ef-357d-4eea-871f-278e81750d7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:10:01.836931 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5507ef-357d-4eea-871f-278e81750d7a admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:10:01.837216 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5507ef-357d-4eea-871f-278e81750d7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:10:01.837526 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5507ef-357d-4eea-871f-278e81750d7a admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:10:01.838298 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5507ef-357d-4eea-871f-278e81750d7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:10:01.840453 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5507ef-357d-4eea-871f-278e81750d7a admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:10:01.851809 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1037/2053] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 15:10:01 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 15:10:01.857210 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb03953c-9680-4e49-8d6c-c505272babf9 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:10:01.868401 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb03953c-9680-4e49-8d6c-c505272babf9 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:10:01.874090 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec409b9-c400-4143-a40d-01331650e69e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:10:01.874960 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec409b9-c400-4143-a40d-01331650e69e admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:10:01.875247 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec409b9-c400-4143-a40d-01331650e69e admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:10:01.875528 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec409b9-c400-4143-a40d-01331650e69e admin admin] PATH_INFO: `/v3/projects/f531692488ad43cf83bea73d4fa07383/users/c08513000c7f43998fb9a95f064e6131/roles/a2e551773cbd4df084020676865b8222` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:10:01.877393 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a310ffa-2ed3-4239-a9f8-ae1afc959865 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:10:01.878085 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a310ffa-2ed3-4239-a9f8-ae1afc959865 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:10:01.878559 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a310ffa-2ed3-4239-a9f8-ae1afc959865 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:10:01.880630 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1038/2054] 10.222.0.46 () {54 vars in 921 bytes} [Thu May 26 15:10:01 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 15:10:01.888442 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6e8cba-9543-49b4-93fc-09a21e5bfd84 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:10:01.889174 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6e8cba-9543-49b4-93fc-09a21e5bfd84 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:10:01.889553 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6e8cba-9543-49b4-93fc-09a21e5bfd84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:10:01.894403 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec409b9-c400-4143-a40d-01331650e69e admin admin] RBAC: Authorizing `identity:create_grant(project_id=f531692488ad43cf83bea73d4fa07383, user_id=c08513000c7f43998fb9a95f064e6131, role_id=a2e551773cbd4df084020676865b8222)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:10:01.897091 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec409b9-c400-4143-a40d-01331650e69e admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:10:01.898692 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:10:01.913835 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1017/2055] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 15:10:01 2022] PUT /identity/v3/projects/f531692488ad43cf83bea73d4fa07383/users/c08513000c7f43998fb9a95f064e6131/roles/a2e551773cbd4df084020676865b8222 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 15:10:01.915285 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-ef6e8cba-9543-49b4-93fc-09a21e5bfd84 None None] MFA Rules not processed for user `bf55a00e3b084aa3b22c25a90e6a773c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:10:01.917003 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-ef6e8cba-9543-49b4-93fc-09a21e5bfd84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:10:01.918208 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec409b9-c400-4143-a40d-01331650e69e admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:10:01.930051 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:10:02.005093 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1039/2056] 10.222.0.46 () {58 vars in 1018 bytes} [Thu May 26 15:10:01 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 15:10:02.005793 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6e8cba-9543-49b4-93fc-09a21e5bfd84 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:10:02.007550 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-ef6e8cba-9543-49b4-93fc-09a21e5bfd84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:10:02.019120 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec409b9-c400-4143-a40d-01331650e69e admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:10:02.024503 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb690044-76bd-4d45-83e2-69b9efc183fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:10:02.024713 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6e8cba-9543-49b4-93fc-09a21e5bfd84 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:10:02.025329 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb690044-76bd-4d45-83e2-69b9efc183fc admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:10:02.025710 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb690044-76bd-4d45-83e2-69b9efc183fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:10:02.026070 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb690044-76bd-4d45-83e2-69b9efc183fc admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:10:02.027767 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb690044-76bd-4d45-83e2-69b9efc183fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:10:02.029603 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb690044-76bd-4d45-83e2-69b9efc183fc admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:10:02.031995 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efeb4f55-74b5-4286-98d4-a7b7646af9c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:10:02.032856 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efeb4f55-74b5-4286-98d4-a7b7646af9c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:10:02.033057 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efeb4f55-74b5-4286-98d4-a7b7646af9c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:10:02.033340 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efeb4f55-74b5-4286-98d4-a7b7646af9c2 admin admin] PATH_INFO: `/v3/users/166bfe89e44847d7b380f33e2e28401d` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:10:02.034703 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efeb4f55-74b5-4286-98d4-a7b7646af9c2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=166bfe89e44847d7b380f33e2e28401d)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:10:02.037190 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efeb4f55-74b5-4286-98d4-a7b7646af9c2 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:10:02.045683 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1018/2057] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 15:10:01 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 15:10:02.047806 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb690044-76bd-4d45-83e2-69b9efc183fc admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:10:02.049376 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-fb690044-76bd-4d45-83e2-69b9efc183fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:10:02.066633 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb690044-76bd-4d45-83e2-69b9efc183fc admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:10:02.071914 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14667aaf-4863-4fe2-b010-4403f2557b26 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf55a00e3b084aa3b22c25a90e6a773c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:10:02.072706 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14667aaf-4863-4fe2-b010-4403f2557b26 service placement] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:10:02.073110 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14667aaf-4863-4fe2-b010-4403f2557b26 service placement] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:10:02.074573 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14667aaf-4863-4fe2-b010-4403f2557b26 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:10:02.085746 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14667aaf-4863-4fe2-b010-4403f2557b26 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:10:02.087415 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14667aaf-4863-4fe2-b010-4403f2557b26 service placement] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:10:02.097022 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-efeb4f55-74b5-4286-98d4-a7b7646af9c2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '166bfe89e44847d7b380f33e2e28401d'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:10:02.100500 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1019/2058] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:10:02 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 15:10:02.107022 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-efeb4f55-74b5-4286-98d4-a7b7646af9c2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '166bfe89e44847d7b380f33e2e28401d'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:10:02.107631 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-efeb4f55-74b5-4286-98d4-a7b7646af9c2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '166bfe89e44847d7b380f33e2e28401d'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:10:02.109884 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14667aaf-4863-4fe2-b010-4403f2557b26 service placement] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:10:02.120869 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14667aaf-4863-4fe2-b010-4403f2557b26 service placement] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:10:02.125595 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-efeb4f55-74b5-4286-98d4-a7b7646af9c2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '166bfe89e44847d7b380f33e2e28401d'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:10:02.126444 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e760f6-ae14-467f-aab1-feadcd185f68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:10:02.127616 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e760f6-ae14-467f-aab1-feadcd185f68 admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:10:02.127974 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e760f6-ae14-467f-aab1-feadcd185f68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:10:02.128388 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e760f6-ae14-467f-aab1-feadcd185f68 admin admin] PATH_INFO: `/v3/projects/f531692488ad43cf83bea73d4fa07383/users/c08513000c7f43998fb9a95f064e6131/roles/a2e551773cbd4df084020676865b8222` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:10:02.130862 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e760f6-ae14-467f-aab1-feadcd185f68 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f531692488ad43cf83bea73d4fa07383, user_id=c08513000c7f43998fb9a95f064e6131, role_id=a2e551773cbd4df084020676865b8222)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:10:02.133914 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e760f6-ae14-467f-aab1-feadcd185f68 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:10:02.149395 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1020/2059] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 15:10:02 2022] PUT /identity/v3/projects/f531692488ad43cf83bea73d4fa07383/users/c08513000c7f43998fb9a95f064e6131/roles/a2e551773cbd4df084020676865b8222 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:10:02.150700 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-efeb4f55-74b5-4286-98d4-a7b7646af9c2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '166bfe89e44847d7b380f33e2e28401d'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:10:02.160349 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd4d7f5-514d-4dde-9e10-8553bb1bf810 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:10:02.160642 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd4d7f5-514d-4dde-9e10-8553bb1bf810 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:10:02.160923 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd4d7f5-514d-4dde-9e10-8553bb1bf810 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:10:02.165902 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1040/2060] 10.222.0.46 () {58 vars in 1223 bytes} [Thu May 26 15:10:02 2022] DELETE /identity/v3/users/166bfe89e44847d7b380f33e2e28401d => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:10:02.199888 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-9fd4d7f5-514d-4dde-9e10-8553bb1bf810 None None] MFA Rules not processed for user `c08513000c7f43998fb9a95f064e6131`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:10:02.200916 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-9fd4d7f5-514d-4dde-9e10-8553bb1bf810 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:10:02.331842 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1021/2061] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:10:02 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 15:10:02.341755 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efeb4f55-74b5-4286-98d4-a7b7646af9c2 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:10:02.347546 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-efeb4f55-74b5-4286-98d4-a7b7646af9c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:10:02.362849 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd4d7f5-514d-4dde-9e10-8553bb1bf810 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:10:02.366806 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-9fd4d7f5-514d-4dde-9e10-8553bb1bf810 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:10:02.430608 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efeb4f55-74b5-4286-98d4-a7b7646af9c2 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:10:02.436063 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a93afbb-f356-4d5f-af5a-c3130bbac1b0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5b5b35614a547d783e6e77ddb8c1c03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:10:02.436816 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a93afbb-f356-4d5f-af5a-c3130bbac1b0 service cinder] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:10:02.437182 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a93afbb-f356-4d5f-af5a-c3130bbac1b0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:10:02.437509 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a93afbb-f356-4d5f-af5a-c3130bbac1b0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:10:02.439492 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-3a93afbb-f356-4d5f-af5a-c3130bbac1b0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:10:02.473587 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:10:02.487929 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd4d7f5-514d-4dde-9e10-8553bb1bf810 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:10:02.506871 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fb97a7-c244-4435-85f2-5eebe5f0b6ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:10:02.506871 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fb97a7-c244-4435-85f2-5eebe5f0b6ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:10:02.506871 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fb97a7-c244-4435-85f2-5eebe5f0b6ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:10:02.506871 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fb97a7-c244-4435-85f2-5eebe5f0b6ce admin admin] PATH_INFO: `/v3/projects/ac537167367948ef88587242df4e921d` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:10:02.507220 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fb97a7-c244-4435-85f2-5eebe5f0b6ce admin admin] RBAC: Authorizing `identity:delete_project(project_id=ac537167367948ef88587242df4e921d)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:10:02.507220 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74fb97a7-c244-4435-85f2-5eebe5f0b6ce admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:10:02.513097 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:10:02.520828 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a93afbb-f356-4d5f-af5a-c3130bbac1b0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:10:02.522473 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a93afbb-f356-4d5f-af5a-c3130bbac1b0 service cinder] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:10:02.532979 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1041/2062] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:10:02 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 15:10:02.565537 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-74fb97a7-c244-4435-85f2-5eebe5f0b6ce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ac537167367948ef88587242df4e921d', 'request_id': 'req-74fb97a7-c244-4435-85f2-5eebe5f0b6ce'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:10:02.572286 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-74fb97a7-c244-4435-85f2-5eebe5f0b6ce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ac537167367948ef88587242df4e921d', 'request_id': 'req-74fb97a7-c244-4435-85f2-5eebe5f0b6ce'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:10:02.578810 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-74fb97a7-c244-4435-85f2-5eebe5f0b6ce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ac537167367948ef88587242df4e921d', 'request_id': 'req-74fb97a7-c244-4435-85f2-5eebe5f0b6ce'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:10:02.591501 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-74fb97a7-c244-4435-85f2-5eebe5f0b6ce admin admin] The token cache is being invalidate because project ac537167367948ef88587242df4e921d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 15:10:02.591787 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-74fb97a7-c244-4435-85f2-5eebe5f0b6ce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:10:02.592440 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-74fb97a7-c244-4435-85f2-5eebe5f0b6ce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:10:02.594041 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1022/2063] 10.222.0.46 () {58 vars in 1232 bytes} [Thu May 26 15:10:02 2022] DELETE /identity/v3/projects/ac537167367948ef88587242df4e921d => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:10:02.599444 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4f7962-4dbc-4488-bfab-075ebd065c33 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:10:02.599747 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4f7962-4dbc-4488-bfab-075ebd065c33 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:10:02.600041 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4f7962-4dbc-4488-bfab-075ebd065c33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:10:02.619619 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-3c4f7962-4dbc-4488-bfab-075ebd065c33 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:10:02.621392 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-3c4f7962-4dbc-4488-bfab-075ebd065c33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:10:02.690108 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1042/2064] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:10:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 15:10:02.699161 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0497d63-a957-4e0f-86b8-fee0cbd50081 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:10:02.699471 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0497d63-a957-4e0f-86b8-fee0cbd50081 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:10:02.699747 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0497d63-a957-4e0f-86b8-fee0cbd50081 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:10:02.721066 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-a0497d63-a957-4e0f-86b8-fee0cbd50081 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:10:02.722312 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-a0497d63-a957-4e0f-86b8-fee0cbd50081 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:10:02.733417 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1023/2065] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:10:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:10:02.737972 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4f7962-4dbc-4488-bfab-075ebd065c33 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:10:02.755299 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4f7962-4dbc-4488-bfab-075ebd065c33 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:10:02.762326 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feae30bf-94ba-4e89-8de7-e8ede1e3bc7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:10:02.763794 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feae30bf-94ba-4e89-8de7-e8ede1e3bc7b admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:10:02.764092 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feae30bf-94ba-4e89-8de7-e8ede1e3bc7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:10:02.764424 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feae30bf-94ba-4e89-8de7-e8ede1e3bc7b admin admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:10:02.765571 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feae30bf-94ba-4e89-8de7-e8ede1e3bc7b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 15:10:02.766297 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feae30bf-94ba-4e89-8de7-e8ede1e3bc7b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:10:02.768416 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feae30bf-94ba-4e89-8de7-e8ede1e3bc7b admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:10:02.781740 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1043/2066] 10.222.0.46 () {58 vars in 1152 bytes} [Thu May 26 15:10:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 15:10:02.786077 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0497d63-a957-4e0f-86b8-fee0cbd50081 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:10:02.797808 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0497d63-a957-4e0f-86b8-fee0cbd50081 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:10:02.810527 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af836a14-5c44-4fa7-8fbc-758a05dfe5c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:10:02.810527 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af836a14-5c44-4fa7-8fbc-758a05dfe5c8 admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:10:02.810527 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af836a14-5c44-4fa7-8fbc-758a05dfe5c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:10:02.810527 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af836a14-5c44-4fa7-8fbc-758a05dfe5c8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:10:02.810901 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af836a14-5c44-4fa7-8fbc-758a05dfe5c8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:10:02.810901 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af836a14-5c44-4fa7-8fbc-758a05dfe5c8 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:10:02.843363 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1024/2067] 10.222.0.46 () {60 vars in 1152 bytes} [Thu May 26 15:10:02 2022] POST /identity/v3/projects => generated 393 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 15:10:02.847333 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feae30bf-94ba-4e89-8de7-e8ede1e3bc7b admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:10:02.867972 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feae30bf-94ba-4e89-8de7-e8ede1e3bc7b admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:10:02.932682 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff35700-d0e3-43cc-8ec4-6095fb037b61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:10:02.934124 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff35700-d0e3-43cc-8ec4-6095fb037b61 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:10:02.934408 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff35700-d0e3-43cc-8ec4-6095fb037b61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:10:02.934698 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff35700-d0e3-43cc-8ec4-6095fb037b61 admin admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:10:02.935391 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff35700-d0e3-43cc-8ec4-6095fb037b61 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:10:02.937443 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff35700-d0e3-43cc-8ec4-6095fb037b61 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:10:02.958548 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1044/2068] 10.222.0.46 () {60 vars in 1143 bytes} [Thu May 26 15:10:02 2022] POST /identity/v3/users => generated 297 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 15:10:02.962839 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af836a14-5c44-4fa7-8fbc-758a05dfe5c8 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:10:02.973861 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af836a14-5c44-4fa7-8fbc-758a05dfe5c8 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:10:02.978743 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e1df9d-2c1d-4490-b377-016884fecd78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:10:02.979432 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e1df9d-2c1d-4490-b377-016884fecd78 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:10:02.979716 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e1df9d-2c1d-4490-b377-016884fecd78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:10:02.980550 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e1df9d-2c1d-4490-b377-016884fecd78 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:10:02.981286 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e1df9d-2c1d-4490-b377-016884fecd78 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:10:02.983005 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e1df9d-2c1d-4490-b377-016884fecd78 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:10:02.995232 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1025/2069] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 15:10:02 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 15:10:03.000268 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff35700-d0e3-43cc-8ec4-6095fb037b61 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:10:03.013123 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff35700-d0e3-43cc-8ec4-6095fb037b61 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:10:03.018346 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cd212f-d59c-44e7-a752-276ae3db7902 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:10:03.018925 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cd212f-d59c-44e7-a752-276ae3db7902 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:10:03.019207 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cd212f-d59c-44e7-a752-276ae3db7902 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:10:03.019487 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cd212f-d59c-44e7-a752-276ae3db7902 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:10:03.020221 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8cd212f-d59c-44e7-a752-276ae3db7902 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:10:03.021914 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8cd212f-d59c-44e7-a752-276ae3db7902 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:10:03.033006 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1045/2070] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 15:10:02 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 15:10:03.037391 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e1df9d-2c1d-4490-b377-016884fecd78 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:10:03.048320 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e1df9d-2c1d-4490-b377-016884fecd78 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:10:03.053159 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ceecb0d-6d96-4deb-9fe4-f84680b97cc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:10:03.054105 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ceecb0d-6d96-4deb-9fe4-f84680b97cc5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:10:03.054385 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ceecb0d-6d96-4deb-9fe4-f84680b97cc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:10:03.054661 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ceecb0d-6d96-4deb-9fe4-f84680b97cc5 admin admin] PATH_INFO: `/v3/projects/45e3fe61cc704411ac645e5935b9d258/users/27172e53a12a437c9d57a75a95727d71/roles/d4088d1845a24fbd9bd8975ce0b1d3d0` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:10:03.071656 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ceecb0d-6d96-4deb-9fe4-f84680b97cc5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=45e3fe61cc704411ac645e5935b9d258, user_id=27172e53a12a437c9d57a75a95727d71, role_id=d4088d1845a24fbd9bd8975ce0b1d3d0)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:10:03.074382 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ceecb0d-6d96-4deb-9fe4-f84680b97cc5 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:10:03.089015 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1026/2071] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 15:10:03 2022] PUT /identity/v3/projects/45e3fe61cc704411ac645e5935b9d258/users/27172e53a12a437c9d57a75a95727d71/roles/d4088d1845a24fbd9bd8975ce0b1d3d0 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:10:03.094848 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66b1227-f291-42ee-ba47-6d09547eaa3d None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:10:03.095141 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66b1227-f291-42ee-ba47-6d09547eaa3d None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:10:03.095426 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66b1227-f291-42ee-ba47-6d09547eaa3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:10:03.114620 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-b66b1227-f291-42ee-ba47-6d09547eaa3d None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:10:03.115858 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-b66b1227-f291-42ee-ba47-6d09547eaa3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:10:03.146259 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:10:03.179441 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1046/2072] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:10:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:10:03.189034 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ceecb0d-6d96-4deb-9fe4-f84680b97cc5 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:10:03.190821 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-1ceecb0d-6d96-4deb-9fe4-f84680b97cc5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:10:03.256211 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ceecb0d-6d96-4deb-9fe4-f84680b97cc5 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:10:03.260103 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff11c51-992f-400a-91ca-ec0b7c838d46 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0156a2f6f3794f479e76b85bdbbe5c26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:10:03.260775 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff11c51-992f-400a-91ca-ec0b7c838d46 service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:10:03.261061 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff11c51-992f-400a-91ca-ec0b7c838d46 service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:10:03.261941 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff11c51-992f-400a-91ca-ec0b7c838d46 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:10:03.271876 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff11c51-992f-400a-91ca-ec0b7c838d46 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:10:03.273345 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff11c51-992f-400a-91ca-ec0b7c838d46 service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:10:03.283856 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1027/2073] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:10:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 15:10:05.279404 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66b1227-f291-42ee-ba47-6d09547eaa3d None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:10:05.281358 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-b66b1227-f291-42ee-ba47-6d09547eaa3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:10:05.402936 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66b1227-f291-42ee-ba47-6d09547eaa3d None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:10:05.407992 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647f1453-36fa-4ade-bf52-edaed7bfe2dd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5b5b35614a547d783e6e77ddb8c1c03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:10:05.408752 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647f1453-36fa-4ade-bf52-edaed7bfe2dd service cinder] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:10:05.409102 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647f1453-36fa-4ade-bf52-edaed7bfe2dd service cinder] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:10:05.409495 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647f1453-36fa-4ade-bf52-edaed7bfe2dd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:10:05.411597 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-647f1453-36fa-4ade-bf52-edaed7bfe2dd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:10:05.498770 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647f1453-36fa-4ade-bf52-edaed7bfe2dd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:10:05.501425 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647f1453-36fa-4ade-bf52-edaed7bfe2dd service cinder] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:10:05.515508 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1047/2074] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:10:05 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 15:10:06.835359 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff11c51-992f-400a-91ca-ec0b7c838d46 service neutron] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:10:06.853145 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff11c51-992f-400a-91ca-ec0b7c838d46 service neutron] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:10:06.867038 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c4c220-663f-4c4b-a616-3d840ba4f2f7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5b5b35614a547d783e6e77ddb8c1c03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:10:06.867712 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c4c220-663f-4c4b-a616-3d840ba4f2f7 service cinder] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:10:06.867997 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c4c220-663f-4c4b-a616-3d840ba4f2f7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:10:06.868275 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c4c220-663f-4c4b-a616-3d840ba4f2f7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:10:06.875923 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-75c4c220-663f-4c4b-a616-3d840ba4f2f7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:10:06.945518 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:10:07.002613 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c4c220-663f-4c4b-a616-3d840ba4f2f7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:10:07.004966 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c4c220-663f-4c4b-a616-3d840ba4f2f7 service cinder] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:10:07.020914 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1028/2075] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:10:06 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 15:10:09.260438 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86de70fa-486f-4c0e-a930-9ce648bed659 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:10:09.260996 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86de70fa-486f-4c0e-a930-9ce648bed659 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:10:09.261096 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86de70fa-486f-4c0e-a930-9ce648bed659 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:10:09.313411 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-86de70fa-486f-4c0e-a930-9ce648bed659 None None] MFA Rules not processed for user `27172e53a12a437c9d57a75a95727d71`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:10:09.314683 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-86de70fa-486f-4c0e-a930-9ce648bed659 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:10:09.397189 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1048/2076] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:10:09 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:10:09.407375 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486a4e73-b6ed-48c4-bb1a-3d5c354ab282 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:10:09.409443 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486a4e73-b6ed-48c4-bb1a-3d5c354ab282 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:10:09.409443 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486a4e73-b6ed-48c4-bb1a-3d5c354ab282 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:10:09.444435 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-486a4e73-b6ed-48c4-bb1a-3d5c354ab282 None None] MFA Rules not processed for user `27172e53a12a437c9d57a75a95727d71`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:10:09.446370 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-486a4e73-b6ed-48c4-bb1a-3d5c354ab282 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:10:09.456474 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1029/2077] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:10:09 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:10:09.470419 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86de70fa-486f-4c0e-a930-9ce648bed659 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:10:09.471968 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-86de70fa-486f-4c0e-a930-9ce648bed659 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:10:09.513665 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:10:09.580683 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86de70fa-486f-4c0e-a930-9ce648bed659 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:10:09.586697 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3e26f8-a085-4354-b95d-30ba6c852eb4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6639353c70449d1b3d10cd6487c3b00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:10:09.587646 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3e26f8-a085-4354-b95d-30ba6c852eb4 service nova] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:10:09.587646 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3e26f8-a085-4354-b95d-30ba6c852eb4 service nova] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:10:09.587927 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3e26f8-a085-4354-b95d-30ba6c852eb4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:10:09.597758 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3e26f8-a085-4354-b95d-30ba6c852eb4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:10:09.599333 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3e26f8-a085-4354-b95d-30ba6c852eb4 service nova] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:10:09.617071 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1049/2078] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:10:09 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 15:10:10.158605 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486a4e73-b6ed-48c4-bb1a-3d5c354ab282 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:10:10.160108 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-486a4e73-b6ed-48c4-bb1a-3d5c354ab282 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:10:10.187574 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486a4e73-b6ed-48c4-bb1a-3d5c354ab282 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:10:10.197144 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbb0dfd2-3bb1-4042-b25d-f4a0d0258be6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:10:10.198081 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb0dfd2-3bb1-4042-b25d-f4a0d0258be6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:10:10.198989 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb0dfd2-3bb1-4042-b25d-f4a0d0258be6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:10:10.199328 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbb0dfd2-3bb1-4042-b25d-f4a0d0258be6 admin admin] PATH_INFO: `/v3/users/5ee7f55d453648fd9af34906b217c5fb` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:10:10.200665 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb0dfd2-3bb1-4042-b25d-f4a0d0258be6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5ee7f55d453648fd9af34906b217c5fb)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:10:10.202533 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbb0dfd2-3bb1-4042-b25d-f4a0d0258be6 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:10:10.295530 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-fbb0dfd2-3bb1-4042-b25d-f4a0d0258be6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5ee7f55d453648fd9af34906b217c5fb'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:10:10.309229 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-fbb0dfd2-3bb1-4042-b25d-f4a0d0258be6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5ee7f55d453648fd9af34906b217c5fb'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:10:10.309956 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-fbb0dfd2-3bb1-4042-b25d-f4a0d0258be6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5ee7f55d453648fd9af34906b217c5fb'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:10:10.344261 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-fbb0dfd2-3bb1-4042-b25d-f4a0d0258be6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5ee7f55d453648fd9af34906b217c5fb'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:10:10.360529 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-fbb0dfd2-3bb1-4042-b25d-f4a0d0258be6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5ee7f55d453648fd9af34906b217c5fb'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:10:10.379136 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1030/2079] 10.222.0.46 () {58 vars in 1223 bytes} [Thu May 26 15:10:10 2022] DELETE /identity/v3/users/5ee7f55d453648fd9af34906b217c5fb => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:10:10.384130 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7139293e-2c67-49d6-8a16-c0e54438a7ea None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:10:10.384646 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7139293e-2c67-49d6-8a16-c0e54438a7ea None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:10:10.384851 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7139293e-2c67-49d6-8a16-c0e54438a7ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:10:10.415062 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ef9e78-0364-4537-8a60-6d8bc9032441 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:10:10.415511 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ef9e78-0364-4537-8a60-6d8bc9032441 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:10:10.415511 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ef9e78-0364-4537-8a60-6d8bc9032441 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:10:10.443351 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-7139293e-2c67-49d6-8a16-c0e54438a7ea None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:10:10.444812 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-7139293e-2c67-49d6-8a16-c0e54438a7ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:10:10.448958 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-93ef9e78-0364-4537-8a60-6d8bc9032441 None None] MFA Rules not processed for user `27172e53a12a437c9d57a75a95727d71`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:10:10.450411 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-93ef9e78-0364-4537-8a60-6d8bc9032441 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:10:10.569987 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1031/2080] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:10:10 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:10:10.578505 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1050/2081] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:10:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:10:10.599976 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ef9e78-0364-4537-8a60-6d8bc9032441 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:10:10.601828 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-93ef9e78-0364-4537-8a60-6d8bc9032441 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:10:10.684998 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:10:10.708446 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ef9e78-0364-4537-8a60-6d8bc9032441 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:10:10.713275 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba81d19a-12dd-42f8-89d7-7f411a5d56ee service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0156a2f6f3794f479e76b85bdbbe5c26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:10:10.714268 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba81d19a-12dd-42f8-89d7-7f411a5d56ee service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:10:10.714656 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba81d19a-12dd-42f8-89d7-7f411a5d56ee service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:10:10.715031 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba81d19a-12dd-42f8-89d7-7f411a5d56ee service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:10:10.727931 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba81d19a-12dd-42f8-89d7-7f411a5d56ee service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:10:10.730243 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba81d19a-12dd-42f8-89d7-7f411a5d56ee service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:10:10.749789 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1032/2082] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:10:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 15:10:11.335989 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7139293e-2c67-49d6-8a16-c0e54438a7ea None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:10:11.337355 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-7139293e-2c67-49d6-8a16-c0e54438a7ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:10:11.358048 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7139293e-2c67-49d6-8a16-c0e54438a7ea None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:10:11.371864 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd25ce3-9f74-4dd5-9193-564d2b8393af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:10:11.373116 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd25ce3-9f74-4dd5-9193-564d2b8393af admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:10:11.373408 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd25ce3-9f74-4dd5-9193-564d2b8393af admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:10:11.373732 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd25ce3-9f74-4dd5-9193-564d2b8393af admin admin] PATH_INFO: `/v3/users/f89c99ce8e76462cad97ee91aa482f4c` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:10:11.375708 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd25ce3-9f74-4dd5-9193-564d2b8393af admin admin] RBAC: Authorizing `identity:delete_user(user_id=f89c99ce8e76462cad97ee91aa482f4c)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:10:11.381509 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fd25ce3-9f74-4dd5-9193-564d2b8393af admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:10:11.476823 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba81d19a-12dd-42f8-89d7-7f411a5d56ee service neutron] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:10:11.479955 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-ba81d19a-12dd-42f8-89d7-7f411a5d56ee service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:10:11.523556 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba81d19a-12dd-42f8-89d7-7f411a5d56ee service neutron] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:10:11.544737 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a4ea1f-e2c0-4729-a5ab-b282b12c6290 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:10:11.545425 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-9fd25ce3-9f74-4dd5-9193-564d2b8393af admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f89c99ce8e76462cad97ee91aa482f4c'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:10:11.548201 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a4ea1f-e2c0-4729-a5ab-b282b12c6290 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:10:11.549002 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a4ea1f-e2c0-4729-a5ab-b282b12c6290 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:10:11.550864 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a4ea1f-e2c0-4729-a5ab-b282b12c6290 admin admin] PATH_INFO: `/v3/projects/98699592822c4bd3ab19bc66ccfd01d1` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:10:11.554417 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a4ea1f-e2c0-4729-a5ab-b282b12c6290 admin admin] RBAC: Authorizing `identity:delete_project(project_id=98699592822c4bd3ab19bc66ccfd01d1)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:10:11.560989 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a4ea1f-e2c0-4729-a5ab-b282b12c6290 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:10:11.571542 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-9fd25ce3-9f74-4dd5-9193-564d2b8393af admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f89c99ce8e76462cad97ee91aa482f4c'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:10:11.572674 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-9fd25ce3-9f74-4dd5-9193-564d2b8393af admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f89c99ce8e76462cad97ee91aa482f4c'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:10:11.614660 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-9fd25ce3-9f74-4dd5-9193-564d2b8393af admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f89c99ce8e76462cad97ee91aa482f4c'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:10:11.633573 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-9fd25ce3-9f74-4dd5-9193-564d2b8393af admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f89c99ce8e76462cad97ee91aa482f4c'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:10:11.642450 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1051/2083] 10.222.0.46 () {58 vars in 1223 bytes} [Thu May 26 15:10:11 2022] DELETE /identity/v3/users/f89c99ce8e76462cad97ee91aa482f4c => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 15:10:11.647983 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443f219e-4240-41f2-88d6-3896f5e7496f None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:10:11.649730 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443f219e-4240-41f2-88d6-3896f5e7496f None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:10:11.649730 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443f219e-4240-41f2-88d6-3896f5e7496f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:10:11.677384 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-443f219e-4240-41f2-88d6-3896f5e7496f None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:10:11.679650 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-443f219e-4240-41f2-88d6-3896f5e7496f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:10:11.692502 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:10:11.695401 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-95a4ea1f-e2c0-4729-a5ab-b282b12c6290 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '98699592822c4bd3ab19bc66ccfd01d1', 'request_id': 'req-95a4ea1f-e2c0-4729-a5ab-b282b12c6290'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:10:11.709668 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-95a4ea1f-e2c0-4729-a5ab-b282b12c6290 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '98699592822c4bd3ab19bc66ccfd01d1', 'request_id': 'req-95a4ea1f-e2c0-4729-a5ab-b282b12c6290'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:10:11.721501 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-95a4ea1f-e2c0-4729-a5ab-b282b12c6290 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '98699592822c4bd3ab19bc66ccfd01d1', 'request_id': 'req-95a4ea1f-e2c0-4729-a5ab-b282b12c6290'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:10:11.750236 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-95a4ea1f-e2c0-4729-a5ab-b282b12c6290 admin admin] The token cache is being invalidate because project 98699592822c4bd3ab19bc66ccfd01d1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 15:10:11.750533 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-95a4ea1f-e2c0-4729-a5ab-b282b12c6290 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:10:11.756685 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-95a4ea1f-e2c0-4729-a5ab-b282b12c6290 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:10:11.758547 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1033/2084] 10.222.0.46 () {58 vars in 1232 bytes} [Thu May 26 15:10:11 2022] DELETE /identity/v3/projects/98699592822c4bd3ab19bc66ccfd01d1 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 15:10:11.765373 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2744aacd-7193-4af1-a7e0-cca845d89783 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:10:11.766005 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2744aacd-7193-4af1-a7e0-cca845d89783 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:10:11.766495 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2744aacd-7193-4af1-a7e0-cca845d89783 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:10:11.770723 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1052/2085] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:10:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:10:11.787650 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443f219e-4240-41f2-88d6-3896f5e7496f None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:10:11.791686 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-443f219e-4240-41f2-88d6-3896f5e7496f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:10:11.804959 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-2744aacd-7193-4af1-a7e0-cca845d89783 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:10:11.806211 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-2744aacd-7193-4af1-a7e0-cca845d89783 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:10:11.839440 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1034/2086] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:10:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:10:11.842875 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3352b7-b7be-4b90-a499-6908590901c0 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:10:11.843353 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3352b7-b7be-4b90-a499-6908590901c0 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:10:11.844150 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3352b7-b7be-4b90-a499-6908590901c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:10:11.873334 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-6c3352b7-b7be-4b90-a499-6908590901c0 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:10:11.875067 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-6c3352b7-b7be-4b90-a499-6908590901c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:10:11.881546 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443f219e-4240-41f2-88d6-3896f5e7496f None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:10:11.887316 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee0de41-aa32-49b5-9dbe-374be3f0fe08 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0156a2f6f3794f479e76b85bdbbe5c26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:10:11.888171 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1035/2087] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:10:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:10:11.889360 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee0de41-aa32-49b5-9dbe-374be3f0fe08 service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:10:11.889720 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee0de41-aa32-49b5-9dbe-374be3f0fe08 service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:10:11.890226 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee0de41-aa32-49b5-9dbe-374be3f0fe08 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:10:11.894844 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3352b7-b7be-4b90-a499-6908590901c0 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:10:11.906991 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee0de41-aa32-49b5-9dbe-374be3f0fe08 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:10:11.910530 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3352b7-b7be-4b90-a499-6908590901c0 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:10:11.915463 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee0de41-aa32-49b5-9dbe-374be3f0fe08 service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:10:11.916626 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2ff59b-cb11-4c7b-9f41-985748dd1f67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:10:11.917320 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2ff59b-cb11-4c7b-9f41-985748dd1f67 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:10:11.917687 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2ff59b-cb11-4c7b-9f41-985748dd1f67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:10:11.917979 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2ff59b-cb11-4c7b-9f41-985748dd1f67 admin admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:10:11.919380 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2ff59b-cb11-4c7b-9f41-985748dd1f67 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 15:10:11.919696 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2ff59b-cb11-4c7b-9f41-985748dd1f67 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:10:11.921630 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b2ff59b-cb11-4c7b-9f41-985748dd1f67 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:10:11.931731 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1053/2088] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:10:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 15:10:11.959398 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1036/2089] 10.222.0.46 () {58 vars in 1152 bytes} [Thu May 26 15:10:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 15:10:11.963140 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee0de41-aa32-49b5-9dbe-374be3f0fe08 service neutron] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:10:11.991079 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee0de41-aa32-49b5-9dbe-374be3f0fe08 service neutron] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:10:11.999062 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdefa5d-49e4-4c07-b0ab-2b540d5d52dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:10:11.999862 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdefa5d-49e4-4c07-b0ab-2b540d5d52dd admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:10:12.000166 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdefa5d-49e4-4c07-b0ab-2b540d5d52dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:10:12.000470 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdefa5d-49e4-4c07-b0ab-2b540d5d52dd admin admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:10:12.001126 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acdefa5d-49e4-4c07-b0ab-2b540d5d52dd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:10:12.003258 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acdefa5d-49e4-4c07-b0ab-2b540d5d52dd admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:10:12.041194 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2ff59b-cb11-4c7b-9f41-985748dd1f67 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:10:12.049806 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1054/2090] 10.222.0.46 () {60 vars in 1152 bytes} [Thu May 26 15:10:11 2022] POST /identity/v3/projects => generated 371 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 15:10:12.055811 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2ff59b-cb11-4c7b-9f41-985748dd1f67 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:10:12.266831 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e035ead8-61b6-4a4d-96ad-898b7d0b73ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:10:12.267537 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e035ead8-61b6-4a4d-96ad-898b7d0b73ef admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:10:12.273998 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e035ead8-61b6-4a4d-96ad-898b7d0b73ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:10:12.274613 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e035ead8-61b6-4a4d-96ad-898b7d0b73ef admin admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:10:12.284250 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e035ead8-61b6-4a4d-96ad-898b7d0b73ef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:10:12.289098 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e035ead8-61b6-4a4d-96ad-898b7d0b73ef admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:10:12.348665 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1037/2091] 10.222.0.46 () {60 vars in 1143 bytes} [Thu May 26 15:10:12 2022] POST /identity/v3/users => generated 286 bytes in 308 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 15:10:12.353219 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdefa5d-49e4-4c07-b0ab-2b540d5d52dd admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:10:12.367366 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdefa5d-49e4-4c07-b0ab-2b540d5d52dd admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:10:12.397945 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c644e079-e520-4927-8a7e-e6de32945655 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:10:12.399117 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c644e079-e520-4927-8a7e-e6de32945655 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:10:12.400123 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c644e079-e520-4927-8a7e-e6de32945655 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:10:12.400552 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c644e079-e520-4927-8a7e-e6de32945655 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:10:12.402338 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c644e079-e520-4927-8a7e-e6de32945655 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:10:12.409344 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c644e079-e520-4927-8a7e-e6de32945655 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:10:12.425512 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1055/2092] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 15:10:12 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 15:10:12.427829 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e035ead8-61b6-4a4d-96ad-898b7d0b73ef admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:10:12.443029 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e035ead8-61b6-4a4d-96ad-898b7d0b73ef admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:10:12.451438 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ebbd0b-e694-4512-8a03-2415be7522f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:10:12.452391 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ebbd0b-e694-4512-8a03-2415be7522f1 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:10:12.452856 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ebbd0b-e694-4512-8a03-2415be7522f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:10:12.453422 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ebbd0b-e694-4512-8a03-2415be7522f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:10:12.454574 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ebbd0b-e694-4512-8a03-2415be7522f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:10:12.457266 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ebbd0b-e694-4512-8a03-2415be7522f1 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:10:12.474029 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1038/2093] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 15:10:12 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 15:10:12.478277 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c644e079-e520-4927-8a7e-e6de32945655 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:10:12.492462 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c644e079-e520-4927-8a7e-e6de32945655 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:10:12.498942 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de081361-68cb-4267-8964-0bfa1fcd9b2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:10:12.499909 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de081361-68cb-4267-8964-0bfa1fcd9b2a admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:10:12.500196 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de081361-68cb-4267-8964-0bfa1fcd9b2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:10:12.500814 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de081361-68cb-4267-8964-0bfa1fcd9b2a admin admin] PATH_INFO: `/v3/projects/f826f3583bc941d9a26edae3aee285ce/users/88ac198b5d484006974f9f3d10728dbd/roles/d4088d1845a24fbd9bd8975ce0b1d3d0` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:10:12.525021 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de081361-68cb-4267-8964-0bfa1fcd9b2a admin admin] RBAC: Authorizing `identity:create_grant(project_id=f826f3583bc941d9a26edae3aee285ce, user_id=88ac198b5d484006974f9f3d10728dbd, role_id=d4088d1845a24fbd9bd8975ce0b1d3d0)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:10:12.533853 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de081361-68cb-4267-8964-0bfa1fcd9b2a admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:10:12.568347 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1056/2094] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 15:10:12 2022] PUT /identity/v3/projects/f826f3583bc941d9a26edae3aee285ce/users/88ac198b5d484006974f9f3d10728dbd/roles/d4088d1845a24fbd9bd8975ce0b1d3d0 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 15:10:12.575480 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b19436-87e5-4bf2-9776-7df3e7d04ff1 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:10:12.577939 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b19436-87e5-4bf2-9776-7df3e7d04ff1 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:10:12.580303 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b19436-87e5-4bf2-9776-7df3e7d04ff1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:10:12.607745 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de081361-68cb-4267-8964-0bfa1fcd9b2a admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:10:12.612325 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-de081361-68cb-4267-8964-0bfa1fcd9b2a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:10:12.686449 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-29b19436-87e5-4bf2-9776-7df3e7d04ff1 None None] MFA Rules not processed for user `88ac198b5d484006974f9f3d10728dbd`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:10:12.689919 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-29b19436-87e5-4bf2-9776-7df3e7d04ff1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:10:12.700902 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:10:12.855876 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de081361-68cb-4267-8964-0bfa1fcd9b2a admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:10:12.857241 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1039/2095] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:10:12 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 287 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:10:12.869734 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b19436-87e5-4bf2-9776-7df3e7d04ff1 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:10:12.872116 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-29b19436-87e5-4bf2-9776-7df3e7d04ff1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:10:12.898397 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0733969-5c27-4fbb-80cd-6c2fcc5d55d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:10:12.899955 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0733969-5c27-4fbb-80cd-6c2fcc5d55d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:10:12.902603 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0733969-5c27-4fbb-80cd-6c2fcc5d55d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:10:12.903328 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0733969-5c27-4fbb-80cd-6c2fcc5d55d7 admin admin] PATH_INFO: `/v3/projects/4ee70304b69f476b9e200149ecc51efe` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:10:12.906063 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0733969-5c27-4fbb-80cd-6c2fcc5d55d7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4ee70304b69f476b9e200149ecc51efe)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:10:12.911553 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0733969-5c27-4fbb-80cd-6c2fcc5d55d7 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:10:12.922582 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:10:13.036874 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b19436-87e5-4bf2-9776-7df3e7d04ff1 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:10:13.074685 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-c0733969-5c27-4fbb-80cd-6c2fcc5d55d7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4ee70304b69f476b9e200149ecc51efe', 'request_id': 'req-c0733969-5c27-4fbb-80cd-6c2fcc5d55d7'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:10:13.085927 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-c0733969-5c27-4fbb-80cd-6c2fcc5d55d7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4ee70304b69f476b9e200149ecc51efe', 'request_id': 'req-c0733969-5c27-4fbb-80cd-6c2fcc5d55d7'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:10:13.093421 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-c0733969-5c27-4fbb-80cd-6c2fcc5d55d7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4ee70304b69f476b9e200149ecc51efe', 'request_id': 'req-c0733969-5c27-4fbb-80cd-6c2fcc5d55d7'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:10:13.108139 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-c0733969-5c27-4fbb-80cd-6c2fcc5d55d7 admin admin] The token cache is being invalidate because project 4ee70304b69f476b9e200149ecc51efe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 15:10:13.108945 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-c0733969-5c27-4fbb-80cd-6c2fcc5d55d7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:10:13.110156 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-c0733969-5c27-4fbb-80cd-6c2fcc5d55d7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:10:13.116411 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1057/2096] 10.222.0.46 () {58 vars in 1232 bytes} [Thu May 26 15:10:12 2022] DELETE /identity/v3/projects/4ee70304b69f476b9e200149ecc51efe => generated 0 bytes in 505 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 15:10:13.117940 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0733969-5c27-4fbb-80cd-6c2fcc5d55d7 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:10:13.120740 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-c0733969-5c27-4fbb-80cd-6c2fcc5d55d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:10:13.183484 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d24d700-2dbc-4788-912f-990bfb08f97c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5b5b35614a547d783e6e77ddb8c1c03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:10:13.187689 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d24d700-2dbc-4788-912f-990bfb08f97c service cinder] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:10:13.188003 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d24d700-2dbc-4788-912f-990bfb08f97c service cinder] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:10:13.188276 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d24d700-2dbc-4788-912f-990bfb08f97c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:10:13.194406 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-4d24d700-2dbc-4788-912f-990bfb08f97c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:10:13.227502 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0733969-5c27-4fbb-80cd-6c2fcc5d55d7 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:10:13.233105 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c50e686-6d88-48ea-9853-373f0bb3a00a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:10:13.235173 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c50e686-6d88-48ea-9853-373f0bb3a00a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:10:13.235430 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c50e686-6d88-48ea-9853-373f0bb3a00a admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:10:13.235671 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c50e686-6d88-48ea-9853-373f0bb3a00a admin admin] PATH_INFO: `/v3/users/9740fe51369e4163b8ad0783030b1209` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:10:13.237109 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c50e686-6d88-48ea-9853-373f0bb3a00a admin admin] RBAC: Authorizing `identity:delete_user(user_id=9740fe51369e4163b8ad0783030b1209)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:10:13.239600 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c50e686-6d88-48ea-9853-373f0bb3a00a admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:10:13.352622 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d24d700-2dbc-4788-912f-990bfb08f97c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:10:13.356687 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d24d700-2dbc-4788-912f-990bfb08f97c service cinder] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:10:13.373845 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-7c50e686-6d88-48ea-9853-373f0bb3a00a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9740fe51369e4163b8ad0783030b1209'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:10:13.379129 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1040/2097] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:10:12 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 510 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 15:10:13.391248 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-7c50e686-6d88-48ea-9853-373f0bb3a00a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9740fe51369e4163b8ad0783030b1209'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:10:13.392003 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-7c50e686-6d88-48ea-9853-373f0bb3a00a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9740fe51369e4163b8ad0783030b1209'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:10:13.423796 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-7c50e686-6d88-48ea-9853-373f0bb3a00a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9740fe51369e4163b8ad0783030b1209'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:10:13.455937 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-7c50e686-6d88-48ea-9853-373f0bb3a00a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9740fe51369e4163b8ad0783030b1209'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:10:13.480246 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1058/2098] 10.222.0.46 () {58 vars in 1223 bytes} [Thu May 26 15:10:13 2022] DELETE /identity/v3/users/9740fe51369e4163b8ad0783030b1209 => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:10:14.464775 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d24d700-2dbc-4788-912f-990bfb08f97c service cinder] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:10:14.477369 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d24d700-2dbc-4788-912f-990bfb08f97c service cinder] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:10:14.547353 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:10:14.623645 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab09738-b356-45d7-97ce-e4441569f02c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:10:14.625164 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab09738-b356-45d7-97ce-e4441569f02c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:10:14.633747 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab09738-b356-45d7-97ce-e4441569f02c admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:10:14.634279 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab09738-b356-45d7-97ce-e4441569f02c admin admin] PATH_INFO: `/v3/projects/004344bc46f04195ad9339a593b0b661` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:10:14.636578 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab09738-b356-45d7-97ce-e4441569f02c admin admin] RBAC: Authorizing `identity:delete_project(project_id=004344bc46f04195ad9339a593b0b661)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:10:14.638735 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab09738-b356-45d7-97ce-e4441569f02c admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:10:14.722220 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-7ab09738-b356-45d7-97ce-e4441569f02c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '004344bc46f04195ad9339a593b0b661', 'request_id': 'req-7ab09738-b356-45d7-97ce-e4441569f02c'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:10:14.739084 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-7ab09738-b356-45d7-97ce-e4441569f02c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '004344bc46f04195ad9339a593b0b661', 'request_id': 'req-7ab09738-b356-45d7-97ce-e4441569f02c'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:10:14.748841 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-7ab09738-b356-45d7-97ce-e4441569f02c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '004344bc46f04195ad9339a593b0b661', 'request_id': 'req-7ab09738-b356-45d7-97ce-e4441569f02c'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:10:14.770439 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-7ab09738-b356-45d7-97ce-e4441569f02c admin admin] The token cache is being invalidate because project 004344bc46f04195ad9339a593b0b661 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 15:10:14.770744 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-7ab09738-b356-45d7-97ce-e4441569f02c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:10:14.771534 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-7ab09738-b356-45d7-97ce-e4441569f02c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:10:14.773791 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1041/2099] 10.222.0.46 () {58 vars in 1232 bytes} [Thu May 26 15:10:14 2022] DELETE /identity/v3/projects/004344bc46f04195ad9339a593b0b661 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:10:14.780334 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c50e686-6d88-48ea-9853-373f0bb3a00a admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:10:14.780334 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-7c50e686-6d88-48ea-9853-373f0bb3a00a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:10:14.797048 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c50e686-6d88-48ea-9853-373f0bb3a00a admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:10:14.806460 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:10:14.909352 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d1a930-b2b5-4332-8dd7-f9841054453d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:10:14.910688 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d1a930-b2b5-4332-8dd7-f9841054453d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:10:14.911118 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d1a930-b2b5-4332-8dd7-f9841054453d admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:10:14.911537 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d1a930-b2b5-4332-8dd7-f9841054453d admin admin] PATH_INFO: `/v3/users/c08513000c7f43998fb9a95f064e6131` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:10:14.913885 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d1a930-b2b5-4332-8dd7-f9841054453d admin admin] RBAC: Authorizing `identity:delete_user(user_id=c08513000c7f43998fb9a95f064e6131)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:10:14.917495 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d1a930-b2b5-4332-8dd7-f9841054453d admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:10:15.020190 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-99d1a930-b2b5-4332-8dd7-f9841054453d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c08513000c7f43998fb9a95f064e6131'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:10:15.034921 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-99d1a930-b2b5-4332-8dd7-f9841054453d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c08513000c7f43998fb9a95f064e6131'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:10:15.035865 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-99d1a930-b2b5-4332-8dd7-f9841054453d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c08513000c7f43998fb9a95f064e6131'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:10:15.058490 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-99d1a930-b2b5-4332-8dd7-f9841054453d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c08513000c7f43998fb9a95f064e6131'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:10:15.073617 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-99d1a930-b2b5-4332-8dd7-f9841054453d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c08513000c7f43998fb9a95f064e6131'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:10:15.090366 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1059/2100] 10.222.0.46 () {58 vars in 1223 bytes} [Thu May 26 15:10:14 2022] DELETE /identity/v3/users/c08513000c7f43998fb9a95f064e6131 => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:10:15.979063 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab09738-b356-45d7-97ce-e4441569f02c admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:10:15.982115 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-7ab09738-b356-45d7-97ce-e4441569f02c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:10:16.005493 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab09738-b356-45d7-97ce-e4441569f02c admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:10:16.071770 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc37da8e-faa3-4641-a785-ebee0fbc4260 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:10:16.072694 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc37da8e-faa3-4641-a785-ebee0fbc4260 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:10:16.073006 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc37da8e-faa3-4641-a785-ebee0fbc4260 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:10:16.073291 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc37da8e-faa3-4641-a785-ebee0fbc4260 admin admin] PATH_INFO: `/v3/projects/f531692488ad43cf83bea73d4fa07383` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:10:16.074567 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc37da8e-faa3-4641-a785-ebee0fbc4260 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f531692488ad43cf83bea73d4fa07383)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:10:16.076499 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc37da8e-faa3-4641-a785-ebee0fbc4260 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:10:16.131286 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:10:16.134122 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-cc37da8e-faa3-4641-a785-ebee0fbc4260 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f531692488ad43cf83bea73d4fa07383', 'request_id': 'req-cc37da8e-faa3-4641-a785-ebee0fbc4260'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:10:16.142499 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-cc37da8e-faa3-4641-a785-ebee0fbc4260 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f531692488ad43cf83bea73d4fa07383', 'request_id': 'req-cc37da8e-faa3-4641-a785-ebee0fbc4260'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:10:16.149508 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-cc37da8e-faa3-4641-a785-ebee0fbc4260 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f531692488ad43cf83bea73d4fa07383', 'request_id': 'req-cc37da8e-faa3-4641-a785-ebee0fbc4260'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:10:16.165817 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-cc37da8e-faa3-4641-a785-ebee0fbc4260 admin admin] The token cache is being invalidate because project f531692488ad43cf83bea73d4fa07383 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 15:10:16.166116 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-cc37da8e-faa3-4641-a785-ebee0fbc4260 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:10:16.167829 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-cc37da8e-faa3-4641-a785-ebee0fbc4260 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:10:16.169634 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1042/2101] 10.222.0.46 () {58 vars in 1232 bytes} [Thu May 26 15:10:15 2022] DELETE /identity/v3/projects/f531692488ad43cf83bea73d4fa07383 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:10:16.176245 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ad30d0-b327-4cb8-9911-680252ba0250 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:10:16.176844 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ad30d0-b327-4cb8-9911-680252ba0250 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:10:16.176844 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ad30d0-b327-4cb8-9911-680252ba0250 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:10:16.196784 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-e5ad30d0-b327-4cb8-9911-680252ba0250 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:10:16.198418 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-e5ad30d0-b327-4cb8-9911-680252ba0250 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:10:16.269380 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1060/2102] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:10:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:10:16.278218 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33999760-2a2c-41f9-8477-31786bfe03d0 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:10:16.278764 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33999760-2a2c-41f9-8477-31786bfe03d0 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:10:16.279417 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33999760-2a2c-41f9-8477-31786bfe03d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:10:16.301983 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-33999760-2a2c-41f9-8477-31786bfe03d0 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:10:16.301983 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-33999760-2a2c-41f9-8477-31786bfe03d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:10:16.315257 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1043/2103] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:10:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:10:16.316096 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ad30d0-b327-4cb8-9911-680252ba0250 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:10:16.327390 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ad30d0-b327-4cb8-9911-680252ba0250 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:10:16.332441 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0cd3bd-c33a-4b57-a716-6d4c12f61c37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:10:16.333219 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0cd3bd-c33a-4b57-a716-6d4c12f61c37 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:10:16.333554 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0cd3bd-c33a-4b57-a716-6d4c12f61c37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:10:16.333836 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0cd3bd-c33a-4b57-a716-6d4c12f61c37 admin admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:10:16.334612 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0cd3bd-c33a-4b57-a716-6d4c12f61c37 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 15:10:16.334922 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0cd3bd-c33a-4b57-a716-6d4c12f61c37 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:10:16.336610 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0cd3bd-c33a-4b57-a716-6d4c12f61c37 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:10:16.350978 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1061/2104] 10.222.0.46 () {58 vars in 1152 bytes} [Thu May 26 15:10:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 15:10:16.357293 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33999760-2a2c-41f9-8477-31786bfe03d0 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:10:16.371258 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33999760-2a2c-41f9-8477-31786bfe03d0 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:10:16.382403 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3186feda-41d9-409c-8b8f-bf632dd8484a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:10:16.383548 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3186feda-41d9-409c-8b8f-bf632dd8484a admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:10:16.383833 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3186feda-41d9-409c-8b8f-bf632dd8484a admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:10:16.386157 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3186feda-41d9-409c-8b8f-bf632dd8484a admin admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:10:16.395375 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3186feda-41d9-409c-8b8f-bf632dd8484a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:10:16.400000 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3186feda-41d9-409c-8b8f-bf632dd8484a admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:10:16.483281 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1044/2105] 10.222.0.46 () {60 vars in 1152 bytes} [Thu May 26 15:10:16 2022] POST /identity/v3/projects => generated 389 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 15:10:16.485054 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0cd3bd-c33a-4b57-a716-6d4c12f61c37 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:10:16.502143 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0cd3bd-c33a-4b57-a716-6d4c12f61c37 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:10:16.529881 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:10:16.607694 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b607add8-7b38-4608-96c8-680d6f787ea7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:10:16.608544 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b607add8-7b38-4608-96c8-680d6f787ea7 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:10:16.608900 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b607add8-7b38-4608-96c8-680d6f787ea7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:10:16.609414 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b607add8-7b38-4608-96c8-680d6f787ea7 admin admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:10:16.610313 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b607add8-7b38-4608-96c8-680d6f787ea7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:10:16.612996 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b607add8-7b38-4608-96c8-680d6f787ea7 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:10:16.650167 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1062/2106] 10.222.0.46 () {60 vars in 1143 bytes} [Thu May 26 15:10:16 2022] POST /identity/v3/users => generated 295 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 15:10:16.654449 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3186feda-41d9-409c-8b8f-bf632dd8484a admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:10:16.668576 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3186feda-41d9-409c-8b8f-bf632dd8484a admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:10:16.677181 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af0511c-9e27-40e9-848b-6a31df9385f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:10:16.678442 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af0511c-9e27-40e9-848b-6a31df9385f4 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:10:16.679037 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af0511c-9e27-40e9-848b-6a31df9385f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:10:16.679493 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af0511c-9e27-40e9-848b-6a31df9385f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:10:16.680404 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af0511c-9e27-40e9-848b-6a31df9385f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:10:16.684241 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af0511c-9e27-40e9-848b-6a31df9385f4 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:10:16.705963 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1045/2107] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 15:10:16 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 15:10:16.709445 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b607add8-7b38-4608-96c8-680d6f787ea7 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:10:16.727563 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b607add8-7b38-4608-96c8-680d6f787ea7 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:10:16.732606 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78fe06c3-6c2c-4fae-9678-128c8646d4d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:10:16.734068 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fe06c3-6c2c-4fae-9678-128c8646d4d3 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:10:16.734358 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fe06c3-6c2c-4fae-9678-128c8646d4d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:10:16.734692 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78fe06c3-6c2c-4fae-9678-128c8646d4d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:10:16.735444 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fe06c3-6c2c-4fae-9678-128c8646d4d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:10:16.737790 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78fe06c3-6c2c-4fae-9678-128c8646d4d3 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:10:16.752063 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1063/2108] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 15:10:16 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 15:10:16.756211 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af0511c-9e27-40e9-848b-6a31df9385f4 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:10:16.783816 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af0511c-9e27-40e9-848b-6a31df9385f4 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:10:16.794176 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e54b6f-5893-41d3-8fc4-2d9bb5de02f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:10:16.795327 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e54b6f-5893-41d3-8fc4-2d9bb5de02f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:10:16.795761 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e54b6f-5893-41d3-8fc4-2d9bb5de02f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:10:16.797209 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e54b6f-5893-41d3-8fc4-2d9bb5de02f1 admin admin] PATH_INFO: `/v3/projects/1dc3721222594b3fa82bdf972888592f/users/7ee3e4c99a6347fc8d95ecda7e74b4ad/roles/d4088d1845a24fbd9bd8975ce0b1d3d0` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:10:16.826006 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e54b6f-5893-41d3-8fc4-2d9bb5de02f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1dc3721222594b3fa82bdf972888592f, user_id=7ee3e4c99a6347fc8d95ecda7e74b4ad, role_id=d4088d1845a24fbd9bd8975ce0b1d3d0)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:10:16.829779 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e54b6f-5893-41d3-8fc4-2d9bb5de02f1 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:10:16.878636 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1046/2109] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 15:10:16 2022] PUT /identity/v3/projects/1dc3721222594b3fa82bdf972888592f/users/7ee3e4c99a6347fc8d95ecda7e74b4ad/roles/d4088d1845a24fbd9bd8975ce0b1d3d0 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:10:16.879531 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514fd1d9-fb85-4019-8c60-e3d5fbfba215 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:10:16.880022 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514fd1d9-fb85-4019-8c60-e3d5fbfba215 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:10:16.880518 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514fd1d9-fb85-4019-8c60-e3d5fbfba215 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:10:17.003162 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-514fd1d9-fb85-4019-8c60-e3d5fbfba215 None None] MFA Rules not processed for user `7ee3e4c99a6347fc8d95ecda7e74b4ad`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:10:17.004789 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-514fd1d9-fb85-4019-8c60-e3d5fbfba215 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:10:17.126772 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1064/2110] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:10:16 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 15:10:17.133837 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e54b6f-5893-41d3-8fc4-2d9bb5de02f1 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:10:17.137167 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-22e54b6f-5893-41d3-8fc4-2d9bb5de02f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:10:17.240935 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e54b6f-5893-41d3-8fc4-2d9bb5de02f1 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:10:17.245590 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c891bbde-8126-4e5e-9743-17d9ec455b9a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5b5b35614a547d783e6e77ddb8c1c03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:10:17.246644 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c891bbde-8126-4e5e-9743-17d9ec455b9a service cinder] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:10:17.247298 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c891bbde-8126-4e5e-9743-17d9ec455b9a service cinder] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:10:17.247666 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c891bbde-8126-4e5e-9743-17d9ec455b9a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:10:17.267566 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c891bbde-8126-4e5e-9743-17d9ec455b9a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:10:17.269055 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c891bbde-8126-4e5e-9743-17d9ec455b9a service cinder] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:10:17.282129 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1047/2111] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:10:17 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 15:10:17.781080 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514fd1d9-fb85-4019-8c60-e3d5fbfba215 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:10:17.783111 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-514fd1d9-fb85-4019-8c60-e3d5fbfba215 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:10:17.892772 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514fd1d9-fb85-4019-8c60-e3d5fbfba215 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:10:17.898285 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00989db6-384b-4b0d-b10b-2b2ae95cc2af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:10:17.899095 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00989db6-384b-4b0d-b10b-2b2ae95cc2af admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:10:17.899377 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00989db6-384b-4b0d-b10b-2b2ae95cc2af admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:10:17.899652 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00989db6-384b-4b0d-b10b-2b2ae95cc2af admin admin] PATH_INFO: `/v3/users/ec18059e0f2943769ae56fbf7a94be6b` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:10:17.901082 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00989db6-384b-4b0d-b10b-2b2ae95cc2af admin admin] RBAC: Authorizing `identity:delete_user(user_id=ec18059e0f2943769ae56fbf7a94be6b)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:10:17.903716 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00989db6-384b-4b0d-b10b-2b2ae95cc2af admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:10:17.949509 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:10:17.975555 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-00989db6-384b-4b0d-b10b-2b2ae95cc2af admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ec18059e0f2943769ae56fbf7a94be6b'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:10:17.988843 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-00989db6-384b-4b0d-b10b-2b2ae95cc2af admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ec18059e0f2943769ae56fbf7a94be6b'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:10:17.989153 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-00989db6-384b-4b0d-b10b-2b2ae95cc2af admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ec18059e0f2943769ae56fbf7a94be6b'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:10:18.009230 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-00989db6-384b-4b0d-b10b-2b2ae95cc2af admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ec18059e0f2943769ae56fbf7a94be6b'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:10:18.021436 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-00989db6-384b-4b0d-b10b-2b2ae95cc2af admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ec18059e0f2943769ae56fbf7a94be6b'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:10:18.029709 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1065/2112] 10.222.0.46 () {58 vars in 1223 bytes} [Thu May 26 15:10:17 2022] DELETE /identity/v3/users/ec18059e0f2943769ae56fbf7a94be6b => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:10:18.258451 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c891bbde-8126-4e5e-9743-17d9ec455b9a service cinder] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:10:18.276083 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c891bbde-8126-4e5e-9743-17d9ec455b9a service cinder] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:10:18.349615 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d91ced4-7e37-4ec8-9c04-7f5fcd223962 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:10:18.350514 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d91ced4-7e37-4ec8-9c04-7f5fcd223962 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:10:18.350800 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d91ced4-7e37-4ec8-9c04-7f5fcd223962 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:10:18.351082 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d91ced4-7e37-4ec8-9c04-7f5fcd223962 admin admin] PATH_INFO: `/v3/projects/75b35b9ac1764016b396142a41ee5d1e` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:10:18.352295 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d91ced4-7e37-4ec8-9c04-7f5fcd223962 admin admin] RBAC: Authorizing `identity:delete_project(project_id=75b35b9ac1764016b396142a41ee5d1e)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:10:18.354269 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d91ced4-7e37-4ec8-9c04-7f5fcd223962 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:10:18.376379 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=94,keystone:DELETE=3,keystone:INSERT=3 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:10:18.416172 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-8d91ced4-7e37-4ec8-9c04-7f5fcd223962 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '75b35b9ac1764016b396142a41ee5d1e', 'request_id': 'req-8d91ced4-7e37-4ec8-9c04-7f5fcd223962'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:10:18.424173 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-8d91ced4-7e37-4ec8-9c04-7f5fcd223962 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '75b35b9ac1764016b396142a41ee5d1e', 'request_id': 'req-8d91ced4-7e37-4ec8-9c04-7f5fcd223962'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:10:18.430902 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-8d91ced4-7e37-4ec8-9c04-7f5fcd223962 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '75b35b9ac1764016b396142a41ee5d1e', 'request_id': 'req-8d91ced4-7e37-4ec8-9c04-7f5fcd223962'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:10:18.444545 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-8d91ced4-7e37-4ec8-9c04-7f5fcd223962 admin admin] The token cache is being invalidate because project 75b35b9ac1764016b396142a41ee5d1e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 15:10:18.444865 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-8d91ced4-7e37-4ec8-9c04-7f5fcd223962 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:10:18.445862 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-8d91ced4-7e37-4ec8-9c04-7f5fcd223962 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:10:18.447718 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1048/2113] 10.222.0.46 () {58 vars in 1232 bytes} [Thu May 26 15:10:18 2022] DELETE /identity/v3/projects/75b35b9ac1764016b396142a41ee5d1e => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:10:18.452342 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00989db6-384b-4b0d-b10b-2b2ae95cc2af admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:10:18.453877 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-00989db6-384b-4b0d-b10b-2b2ae95cc2af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:10:18.467493 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00989db6-384b-4b0d-b10b-2b2ae95cc2af admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:10:18.531644 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fff464b-bfe1-4998-9951-d807141603fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:10:18.532646 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fff464b-bfe1-4998-9951-d807141603fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:10:18.532936 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fff464b-bfe1-4998-9951-d807141603fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:10:18.533232 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fff464b-bfe1-4998-9951-d807141603fa admin admin] PATH_INFO: `/v3/users/758e9872b34e4aa1aa64c3773ec62b68` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:10:18.534770 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fff464b-bfe1-4998-9951-d807141603fa admin admin] RBAC: Authorizing `identity:delete_user(user_id=758e9872b34e4aa1aa64c3773ec62b68)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:10:18.536876 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fff464b-bfe1-4998-9951-d807141603fa admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:10:18.601593 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-4fff464b-bfe1-4998-9951-d807141603fa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '758e9872b34e4aa1aa64c3773ec62b68'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:10:18.626424 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-4fff464b-bfe1-4998-9951-d807141603fa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '758e9872b34e4aa1aa64c3773ec62b68'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:10:18.628486 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-4fff464b-bfe1-4998-9951-d807141603fa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '758e9872b34e4aa1aa64c3773ec62b68'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:10:18.652146 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-4fff464b-bfe1-4998-9951-d807141603fa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '758e9872b34e4aa1aa64c3773ec62b68'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:10:18.661154 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-4fff464b-bfe1-4998-9951-d807141603fa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '758e9872b34e4aa1aa64c3773ec62b68'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:10:18.669520 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1066/2114] 10.222.0.46 () {58 vars in 1223 bytes} [Thu May 26 15:10:18 2022] DELETE /identity/v3/users/758e9872b34e4aa1aa64c3773ec62b68 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:10:18.948718 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d91ced4-7e37-4ec8-9c04-7f5fcd223962 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:10:18.950285 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-8d91ced4-7e37-4ec8-9c04-7f5fcd223962 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:10:18.977754 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d91ced4-7e37-4ec8-9c04-7f5fcd223962 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:10:19.097333 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bc745e-d2eb-4564-b451-6502796649f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:10:19.098800 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bc745e-d2eb-4564-b451-6502796649f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:10:19.099401 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bc745e-d2eb-4564-b451-6502796649f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:10:19.099848 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bc745e-d2eb-4564-b451-6502796649f8 admin admin] PATH_INFO: `/v3/projects/4b9f0b7e0b90413eaf37e607cac8fd54` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:10:19.101631 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bc745e-d2eb-4564-b451-6502796649f8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4b9f0b7e0b90413eaf37e607cac8fd54)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:10:19.104425 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57bc745e-d2eb-4564-b451-6502796649f8 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:10:19.187160 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-57bc745e-d2eb-4564-b451-6502796649f8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4b9f0b7e0b90413eaf37e607cac8fd54', 'request_id': 'req-57bc745e-d2eb-4564-b451-6502796649f8'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:10:19.191705 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-57bc745e-d2eb-4564-b451-6502796649f8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4b9f0b7e0b90413eaf37e607cac8fd54', 'request_id': 'req-57bc745e-d2eb-4564-b451-6502796649f8'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:10:19.204681 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-57bc745e-d2eb-4564-b451-6502796649f8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4b9f0b7e0b90413eaf37e607cac8fd54', 'request_id': 'req-57bc745e-d2eb-4564-b451-6502796649f8'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:10:19.246901 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-57bc745e-d2eb-4564-b451-6502796649f8 admin admin] The token cache is being invalidate because project 4b9f0b7e0b90413eaf37e607cac8fd54 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 15:10:19.247263 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-57bc745e-d2eb-4564-b451-6502796649f8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:10:19.251037 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-57bc745e-d2eb-4564-b451-6502796649f8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:10:19.256424 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1049/2115] 10.222.0.46 () {58 vars in 1232 bytes} [Thu May 26 15:10:18 2022] DELETE /identity/v3/projects/4b9f0b7e0b90413eaf37e607cac8fd54 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:10:19.263636 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6202298-bf28-4690-9a3d-ce47e44ccbac None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:10:19.265073 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6202298-bf28-4690-9a3d-ce47e44ccbac None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:10:19.265366 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6202298-bf28-4690-9a3d-ce47e44ccbac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:10:19.293600 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-e6202298-bf28-4690-9a3d-ce47e44ccbac None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:10:19.295274 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-e6202298-bf28-4690-9a3d-ce47e44ccbac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:10:19.335335 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:10:19.425568 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1067/2116] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:10:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:10:19.435228 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9435e29b-7142-4cf8-b31a-bdc9836c5aee None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:10:19.435598 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9435e29b-7142-4cf8-b31a-bdc9836c5aee None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:10:19.435598 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9435e29b-7142-4cf8-b31a-bdc9836c5aee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:10:19.454723 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-9435e29b-7142-4cf8-b31a-bdc9836c5aee None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:10:19.456004 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-9435e29b-7142-4cf8-b31a-bdc9836c5aee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:10:19.464148 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1050/2117] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:10:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:10:19.468284 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6202298-bf28-4690-9a3d-ce47e44ccbac None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:10:19.478200 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6202298-bf28-4690-9a3d-ce47e44ccbac None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:10:19.483772 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3287010-cb8e-4e2f-a32c-e84ca6c165ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:10:19.484099 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3287010-cb8e-4e2f-a32c-e84ca6c165ee admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:10:19.484383 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3287010-cb8e-4e2f-a32c-e84ca6c165ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:10:19.484661 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3287010-cb8e-4e2f-a32c-e84ca6c165ee admin admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:10:19.485392 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3287010-cb8e-4e2f-a32c-e84ca6c165ee admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 15:10:19.485736 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3287010-cb8e-4e2f-a32c-e84ca6c165ee admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:10:19.487337 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3287010-cb8e-4e2f-a32c-e84ca6c165ee admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:10:19.500576 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1068/2118] 10.222.0.46 () {58 vars in 1152 bytes} [Thu May 26 15:10:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 15:10:19.504987 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9435e29b-7142-4cf8-b31a-bdc9836c5aee None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:10:19.514638 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9435e29b-7142-4cf8-b31a-bdc9836c5aee None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:10:19.519467 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7861f3-ba99-40f9-9e2e-f5557e65bda5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:10:19.520124 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7861f3-ba99-40f9-9e2e-f5557e65bda5 admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:10:19.520415 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7861f3-ba99-40f9-9e2e-f5557e65bda5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:10:19.520692 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7861f3-ba99-40f9-9e2e-f5557e65bda5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:10:19.521389 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7861f3-ba99-40f9-9e2e-f5557e65bda5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:10:19.523318 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a7861f3-ba99-40f9-9e2e-f5557e65bda5 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:10:19.553655 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1051/2119] 10.222.0.46 () {60 vars in 1152 bytes} [Thu May 26 15:10:19 2022] POST /identity/v3/projects => generated 373 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 15:10:19.554755 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3287010-cb8e-4e2f-a32c-e84ca6c165ee admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:10:19.565675 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3287010-cb8e-4e2f-a32c-e84ca6c165ee admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:10:19.647541 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9807794c-5345-4587-a94e-c5914428366b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:10:19.648148 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9807794c-5345-4587-a94e-c5914428366b admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:10:19.648430 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9807794c-5345-4587-a94e-c5914428366b admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:10:19.648711 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9807794c-5345-4587-a94e-c5914428366b admin admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:10:19.649413 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9807794c-5345-4587-a94e-c5914428366b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:10:19.651945 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9807794c-5345-4587-a94e-c5914428366b admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:10:19.675286 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1069/2120] 10.222.0.46 () {60 vars in 1143 bytes} [Thu May 26 15:10:19 2022] POST /identity/v3/users => generated 287 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 15:10:19.679153 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7861f3-ba99-40f9-9e2e-f5557e65bda5 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:10:19.690502 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7861f3-ba99-40f9-9e2e-f5557e65bda5 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:10:19.695676 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa49a01c-98ea-4acc-a28b-0c5f1d1e3a5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:10:19.696350 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa49a01c-98ea-4acc-a28b-0c5f1d1e3a5a admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:10:19.696637 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa49a01c-98ea-4acc-a28b-0c5f1d1e3a5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:10:19.696917 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa49a01c-98ea-4acc-a28b-0c5f1d1e3a5a admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:10:19.697680 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa49a01c-98ea-4acc-a28b-0c5f1d1e3a5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:10:19.699412 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa49a01c-98ea-4acc-a28b-0c5f1d1e3a5a admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:10:19.700900 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:10:19.713201 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1052/2121] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 15:10:19 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 15:10:19.718195 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9807794c-5345-4587-a94e-c5914428366b admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:10:19.728456 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9807794c-5345-4587-a94e-c5914428366b admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:10:19.735358 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba88778f-1dbc-44d0-a077-0e608d59d66d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:10:19.736046 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba88778f-1dbc-44d0-a077-0e608d59d66d admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:10:19.736374 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba88778f-1dbc-44d0-a077-0e608d59d66d admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:10:19.736670 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba88778f-1dbc-44d0-a077-0e608d59d66d admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:10:19.737424 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba88778f-1dbc-44d0-a077-0e608d59d66d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:10:19.739387 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba88778f-1dbc-44d0-a077-0e608d59d66d admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:10:19.751009 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1070/2122] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 15:10:19 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 15:10:19.754950 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa49a01c-98ea-4acc-a28b-0c5f1d1e3a5a admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:10:19.766594 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa49a01c-98ea-4acc-a28b-0c5f1d1e3a5a admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:10:19.774266 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09e59bf-4479-4d73-ae41-09edc402e9dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:10:19.775161 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09e59bf-4479-4d73-ae41-09edc402e9dd admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:10:19.775446 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09e59bf-4479-4d73-ae41-09edc402e9dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:10:19.775726 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09e59bf-4479-4d73-ae41-09edc402e9dd admin admin] PATH_INFO: `/v3/projects/b4b6730db99340e199be4c1becc47b47/users/6bc99c5667304bae9b1c21e15f2a21f3/roles/d4088d1845a24fbd9bd8975ce0b1d3d0` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:10:19.792779 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09e59bf-4479-4d73-ae41-09edc402e9dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=b4b6730db99340e199be4c1becc47b47, user_id=6bc99c5667304bae9b1c21e15f2a21f3, role_id=d4088d1845a24fbd9bd8975ce0b1d3d0)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:10:19.795803 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09e59bf-4479-4d73-ae41-09edc402e9dd admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:10:19.812231 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1053/2123] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 15:10:19 2022] PUT /identity/v3/projects/b4b6730db99340e199be4c1becc47b47/users/6bc99c5667304bae9b1c21e15f2a21f3/roles/d4088d1845a24fbd9bd8975ce0b1d3d0 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:10:19.822631 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c56da1-8b63-4edb-95d5-a63170dbafcd None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:10:19.822964 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c56da1-8b63-4edb-95d5-a63170dbafcd None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:10:19.823290 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c56da1-8b63-4edb-95d5-a63170dbafcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:10:19.853969 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-a6c56da1-8b63-4edb-95d5-a63170dbafcd None None] MFA Rules not processed for user `6bc99c5667304bae9b1c21e15f2a21f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:10:19.855899 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-a6c56da1-8b63-4edb-95d5-a63170dbafcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:10:19.925757 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1071/2124] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:10:19 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:10:19.930365 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09e59bf-4479-4d73-ae41-09edc402e9dd admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:10:20.004270 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09e59bf-4479-4d73-ae41-09edc402e9dd admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:10:20.009043 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bdf3db-444a-4dba-8f2b-9394b1039ced admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:10:20.009770 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bdf3db-444a-4dba-8f2b-9394b1039ced admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:10:20.010056 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bdf3db-444a-4dba-8f2b-9394b1039ced admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:10:20.010889 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bdf3db-444a-4dba-8f2b-9394b1039ced admin admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:10:20.011593 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6bdf3db-444a-4dba-8f2b-9394b1039ced admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:10:20.013542 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6bdf3db-444a-4dba-8f2b-9394b1039ced admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:10:20.043064 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1054/2125] 10.222.0.46 () {60 vars in 1152 bytes} [Thu May 26 15:10:19 2022] POST /identity/v3/projects => generated 373 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 15:10:20.044323 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c56da1-8b63-4edb-95d5-a63170dbafcd None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:10:20.055391 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c56da1-8b63-4edb-95d5-a63170dbafcd None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:10:20.078389 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:10:20.130011 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c6087d-2619-46b2-9db2-c71e4797d294 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:10:20.131148 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c6087d-2619-46b2-9db2-c71e4797d294 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:10:20.131699 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c6087d-2619-46b2-9db2-c71e4797d294 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:10:20.132165 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c6087d-2619-46b2-9db2-c71e4797d294 admin admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:10:20.133047 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c6087d-2619-46b2-9db2-c71e4797d294 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:10:20.136034 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c6087d-2619-46b2-9db2-c71e4797d294 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:10:20.164812 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1072/2126] 10.222.0.46 () {60 vars in 1143 bytes} [Thu May 26 15:10:20 2022] POST /identity/v3/users => generated 293 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 15:10:20.168715 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bdf3db-444a-4dba-8f2b-9394b1039ced admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:10:20.180053 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bdf3db-444a-4dba-8f2b-9394b1039ced admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:10:20.185424 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095446f2-a03e-4aea-b27c-af8b59521b08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:10:20.186717 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095446f2-a03e-4aea-b27c-af8b59521b08 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:10:20.187209 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095446f2-a03e-4aea-b27c-af8b59521b08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:10:20.187494 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095446f2-a03e-4aea-b27c-af8b59521b08 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:10:20.188258 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095446f2-a03e-4aea-b27c-af8b59521b08 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:10:20.189981 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095446f2-a03e-4aea-b27c-af8b59521b08 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:10:20.202436 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1055/2127] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 15:10:20 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 15:10:20.206749 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c6087d-2619-46b2-9db2-c71e4797d294 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:10:20.218386 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c6087d-2619-46b2-9db2-c71e4797d294 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:10:20.223919 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdaa501-9273-4959-aa4b-824b75c06115 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:10:20.224796 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdaa501-9273-4959-aa4b-824b75c06115 admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:10:20.225126 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdaa501-9273-4959-aa4b-824b75c06115 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:10:20.225362 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdaa501-9273-4959-aa4b-824b75c06115 admin admin] PATH_INFO: `/v3/projects/5c81ad15cca24d0fab87a925dee9df66/users/65f5f02a4ea748b795824642b2b7d7f1/roles/a2e551773cbd4df084020676865b8222` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:10:20.249852 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bdaa501-9273-4959-aa4b-824b75c06115 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5c81ad15cca24d0fab87a925dee9df66, user_id=65f5f02a4ea748b795824642b2b7d7f1, role_id=a2e551773cbd4df084020676865b8222)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:10:20.253928 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bdaa501-9273-4959-aa4b-824b75c06115 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:10:20.272742 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1073/2128] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 15:10:20 2022] PUT /identity/v3/projects/5c81ad15cca24d0fab87a925dee9df66/users/65f5f02a4ea748b795824642b2b7d7f1/roles/a2e551773cbd4df084020676865b8222 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:10:20.277160 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095446f2-a03e-4aea-b27c-af8b59521b08 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:10:20.292342 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095446f2-a03e-4aea-b27c-af8b59521b08 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:10:20.361428 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74bde95-258d-4f65-83cc-7b33d9def741 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:10:20.362220 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74bde95-258d-4f65-83cc-7b33d9def741 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:10:20.362412 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74bde95-258d-4f65-83cc-7b33d9def741 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:10:20.362692 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74bde95-258d-4f65-83cc-7b33d9def741 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:10:20.363429 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74bde95-258d-4f65-83cc-7b33d9def741 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:10:20.365055 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74bde95-258d-4f65-83cc-7b33d9def741 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:10:20.376790 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1056/2129] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 15:10:20 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 15:10:20.380980 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdaa501-9273-4959-aa4b-824b75c06115 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:10:20.392325 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdaa501-9273-4959-aa4b-824b75c06115 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:10:20.397167 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2b5e31-2952-4bc7-aa51-c2d42204b9d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:10:20.398167 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2b5e31-2952-4bc7-aa51-c2d42204b9d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:10:20.398520 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2b5e31-2952-4bc7-aa51-c2d42204b9d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:10:20.398865 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2b5e31-2952-4bc7-aa51-c2d42204b9d9 admin admin] PATH_INFO: `/v3/projects/5c81ad15cca24d0fab87a925dee9df66/users/65f5f02a4ea748b795824642b2b7d7f1/roles/a2e551773cbd4df084020676865b8222` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:10:20.400952 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2b5e31-2952-4bc7-aa51-c2d42204b9d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5c81ad15cca24d0fab87a925dee9df66, user_id=65f5f02a4ea748b795824642b2b7d7f1, role_id=a2e551773cbd4df084020676865b8222)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:10:20.403702 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2b5e31-2952-4bc7-aa51-c2d42204b9d9 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:10:20.416380 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1074/2130] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 15:10:20 2022] PUT /identity/v3/projects/5c81ad15cca24d0fab87a925dee9df66/users/65f5f02a4ea748b795824642b2b7d7f1/roles/a2e551773cbd4df084020676865b8222 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 15:10:20.427128 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc33e34-9855-4dfd-b7d9-1b70e92a05ac None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:10:20.427726 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc33e34-9855-4dfd-b7d9-1b70e92a05ac None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:10:20.429196 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc33e34-9855-4dfd-b7d9-1b70e92a05ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:10:20.461307 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-7dc33e34-9855-4dfd-b7d9-1b70e92a05ac None None] MFA Rules not processed for user `65f5f02a4ea748b795824642b2b7d7f1`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:10:20.462813 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-7dc33e34-9855-4dfd-b7d9-1b70e92a05ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:10:20.504110 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:10:20.552882 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1057/2131] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:10:20 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:10:20.567092 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2b5e31-2952-4bc7-aa51-c2d42204b9d9 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:10:20.570330 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-dc2b5e31-2952-4bc7-aa51-c2d42204b9d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:10:20.643396 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2b5e31-2952-4bc7-aa51-c2d42204b9d9 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:10:20.647530 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9544b8bc-f1e8-4e98-a075-58e9f208041e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5b5b35614a547d783e6e77ddb8c1c03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:10:20.648195 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9544b8bc-f1e8-4e98-a075-58e9f208041e service cinder] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:10:20.648482 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9544b8bc-f1e8-4e98-a075-58e9f208041e service cinder] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:10:20.648764 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9544b8bc-f1e8-4e98-a075-58e9f208041e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:10:20.710464 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9544b8bc-f1e8-4e98-a075-58e9f208041e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:10:20.711925 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9544b8bc-f1e8-4e98-a075-58e9f208041e service cinder] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:10:20.726791 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1075/2132] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:10:20 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 15:10:24.358644 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc33e34-9855-4dfd-b7d9-1b70e92a05ac None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:10:24.439362 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc33e34-9855-4dfd-b7d9-1b70e92a05ac None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:10:24.443913 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a02953b-b2e5-40c3-88a5-660a1ac76661 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:10:24.444795 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a02953b-b2e5-40c3-88a5-660a1ac76661 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:10:24.445095 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a02953b-b2e5-40c3-88a5-660a1ac76661 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:10:24.445381 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a02953b-b2e5-40c3-88a5-660a1ac76661 admin admin] PATH_INFO: `/v3/users/6bc99c5667304bae9b1c21e15f2a21f3` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:10:24.446731 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a02953b-b2e5-40c3-88a5-660a1ac76661 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6bc99c5667304bae9b1c21e15f2a21f3)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:10:24.449458 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a02953b-b2e5-40c3-88a5-660a1ac76661 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:10:24.502681 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-5a02953b-b2e5-40c3-88a5-660a1ac76661 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6bc99c5667304bae9b1c21e15f2a21f3'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:10:24.511272 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-5a02953b-b2e5-40c3-88a5-660a1ac76661 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6bc99c5667304bae9b1c21e15f2a21f3'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:10:24.511859 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-5a02953b-b2e5-40c3-88a5-660a1ac76661 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6bc99c5667304bae9b1c21e15f2a21f3'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:10:24.524652 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-5a02953b-b2e5-40c3-88a5-660a1ac76661 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6bc99c5667304bae9b1c21e15f2a21f3'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:10:24.532024 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-5a02953b-b2e5-40c3-88a5-660a1ac76661 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6bc99c5667304bae9b1c21e15f2a21f3'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:10:24.540436 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1058/2133] 10.222.0.46 () {58 vars in 1223 bytes} [Thu May 26 15:10:24 2022] DELETE /identity/v3/users/6bc99c5667304bae9b1c21e15f2a21f3 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:10:24.545182 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6156c5-6b7a-48ca-ac0f-71a1478f3eca None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:10:24.545863 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6156c5-6b7a-48ca-ac0f-71a1478f3eca None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:10:24.545863 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6156c5-6b7a-48ca-ac0f-71a1478f3eca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:10:24.562990 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-6b6156c5-6b7a-48ca-ac0f-71a1478f3eca None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:10:24.564261 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-6b6156c5-6b7a-48ca-ac0f-71a1478f3eca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:10:24.586253 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:10:24.630598 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1076/2134] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:10:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:10:24.642019 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a02953b-b2e5-40c3-88a5-660a1ac76661 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:10:24.643762 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-5a02953b-b2e5-40c3-88a5-660a1ac76661 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:10:24.701472 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:10:24.709386 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a02953b-b2e5-40c3-88a5-660a1ac76661 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:10:24.714561 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30418791-1a6e-4a32-bd45-083ec77b0ac0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0156a2f6f3794f479e76b85bdbbe5c26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:10:24.715323 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30418791-1a6e-4a32-bd45-083ec77b0ac0 service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:10:24.715779 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30418791-1a6e-4a32-bd45-083ec77b0ac0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:10:24.716231 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30418791-1a6e-4a32-bd45-083ec77b0ac0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:10:24.725689 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30418791-1a6e-4a32-bd45-083ec77b0ac0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:10:24.728066 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30418791-1a6e-4a32-bd45-083ec77b0ac0 service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:10:24.739312 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1059/2135] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:10:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 15:10:25.048094 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6156c5-6b7a-48ca-ac0f-71a1478f3eca None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:10:25.049847 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-6b6156c5-6b7a-48ca-ac0f-71a1478f3eca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:10:25.063113 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6156c5-6b7a-48ca-ac0f-71a1478f3eca None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:10:25.068886 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83da47de-7082-4898-b41c-c852310bf229 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:10:25.069962 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83da47de-7082-4898-b41c-c852310bf229 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:10:25.070251 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83da47de-7082-4898-b41c-c852310bf229 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:10:25.070532 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83da47de-7082-4898-b41c-c852310bf229 admin admin] PATH_INFO: `/v3/projects/b4b6730db99340e199be4c1becc47b47` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:10:25.071907 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83da47de-7082-4898-b41c-c852310bf229 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b4b6730db99340e199be4c1becc47b47)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:10:25.074562 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83da47de-7082-4898-b41c-c852310bf229 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:10:25.133428 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-83da47de-7082-4898-b41c-c852310bf229 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b4b6730db99340e199be4c1becc47b47', 'request_id': 'req-83da47de-7082-4898-b41c-c852310bf229'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:10:25.144981 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-83da47de-7082-4898-b41c-c852310bf229 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b4b6730db99340e199be4c1becc47b47', 'request_id': 'req-83da47de-7082-4898-b41c-c852310bf229'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:10:25.154146 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-83da47de-7082-4898-b41c-c852310bf229 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b4b6730db99340e199be4c1becc47b47', 'request_id': 'req-83da47de-7082-4898-b41c-c852310bf229'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:10:25.172814 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-83da47de-7082-4898-b41c-c852310bf229 admin admin] The token cache is being invalidate because project b4b6730db99340e199be4c1becc47b47 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 15:10:25.173281 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-83da47de-7082-4898-b41c-c852310bf229 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:10:25.174310 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-83da47de-7082-4898-b41c-c852310bf229 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:10:25.176780 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1077/2136] 10.222.0.46 () {58 vars in 1232 bytes} [Thu May 26 15:10:25 2022] DELETE /identity/v3/projects/b4b6730db99340e199be4c1becc47b47 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:10:25.180945 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30418791-1a6e-4a32-bd45-083ec77b0ac0 service neutron] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:10:25.192360 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30418791-1a6e-4a32-bd45-083ec77b0ac0 service neutron] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:10:25.260632 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8cd7b33-f9e4-4731-b550-f17d0fbce0ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:10:25.261556 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cd7b33-f9e4-4731-b550-f17d0fbce0ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:10:25.261847 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cd7b33-f9e4-4731-b550-f17d0fbce0ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:10:25.262127 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8cd7b33-f9e4-4731-b550-f17d0fbce0ba admin admin] PATH_INFO: `/v3/users/65f5f02a4ea748b795824642b2b7d7f1` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:10:25.263365 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8cd7b33-f9e4-4731-b550-f17d0fbce0ba admin admin] RBAC: Authorizing `identity:delete_user(user_id=65f5f02a4ea748b795824642b2b7d7f1)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:10:25.265800 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8cd7b33-f9e4-4731-b550-f17d0fbce0ba admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:10:25.324549 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-c8cd7b33-f9e4-4731-b550-f17d0fbce0ba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '65f5f02a4ea748b795824642b2b7d7f1'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:10:25.334653 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-c8cd7b33-f9e4-4731-b550-f17d0fbce0ba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '65f5f02a4ea748b795824642b2b7d7f1'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:10:25.335377 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-c8cd7b33-f9e4-4731-b550-f17d0fbce0ba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '65f5f02a4ea748b795824642b2b7d7f1'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:10:25.350400 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-c8cd7b33-f9e4-4731-b550-f17d0fbce0ba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '65f5f02a4ea748b795824642b2b7d7f1'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:10:25.358897 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-c8cd7b33-f9e4-4731-b550-f17d0fbce0ba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '65f5f02a4ea748b795824642b2b7d7f1'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:10:25.366230 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1060/2137] 10.222.0.46 () {58 vars in 1223 bytes} [Thu May 26 15:10:25 2022] DELETE /identity/v3/users/65f5f02a4ea748b795824642b2b7d7f1 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:10:25.659679 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83da47de-7082-4898-b41c-c852310bf229 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:10:25.662364 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-83da47de-7082-4898-b41c-c852310bf229 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:10:25.696404 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83da47de-7082-4898-b41c-c852310bf229 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:10:25.762884 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01a0c78-1db4-42a7-911b-503eb6e62b68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:10:25.763749 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01a0c78-1db4-42a7-911b-503eb6e62b68 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:10:25.764027 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01a0c78-1db4-42a7-911b-503eb6e62b68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:10:25.764299 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01a0c78-1db4-42a7-911b-503eb6e62b68 admin admin] PATH_INFO: `/v3/projects/5c81ad15cca24d0fab87a925dee9df66` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:10:25.766083 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01a0c78-1db4-42a7-911b-503eb6e62b68 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5c81ad15cca24d0fab87a925dee9df66)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:10:25.767947 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01a0c78-1db4-42a7-911b-503eb6e62b68 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:10:25.816263 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-c01a0c78-1db4-42a7-911b-503eb6e62b68 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5c81ad15cca24d0fab87a925dee9df66', 'request_id': 'req-c01a0c78-1db4-42a7-911b-503eb6e62b68'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:10:25.818761 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:10:25.824062 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-c01a0c78-1db4-42a7-911b-503eb6e62b68 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5c81ad15cca24d0fab87a925dee9df66', 'request_id': 'req-c01a0c78-1db4-42a7-911b-503eb6e62b68'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:10:25.831066 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-c01a0c78-1db4-42a7-911b-503eb6e62b68 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5c81ad15cca24d0fab87a925dee9df66', 'request_id': 'req-c01a0c78-1db4-42a7-911b-503eb6e62b68'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:10:25.842787 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-c01a0c78-1db4-42a7-911b-503eb6e62b68 admin admin] The token cache is being invalidate because project 5c81ad15cca24d0fab87a925dee9df66 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 15:10:25.843073 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-c01a0c78-1db4-42a7-911b-503eb6e62b68 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:10:25.843756 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-c01a0c78-1db4-42a7-911b-503eb6e62b68 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:10:25.845303 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1078/2138] 10.222.0.46 () {58 vars in 1232 bytes} [Thu May 26 15:10:25 2022] DELETE /identity/v3/projects/5c81ad15cca24d0fab87a925dee9df66 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:10:25.850158 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3466b25a-1cb7-4e6a-b387-f2b99834c6a6 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:10:25.850459 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3466b25a-1cb7-4e6a-b387-f2b99834c6a6 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:10:25.850743 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3466b25a-1cb7-4e6a-b387-f2b99834c6a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:10:25.870612 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-3466b25a-1cb7-4e6a-b387-f2b99834c6a6 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:10:25.871821 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-3466b25a-1cb7-4e6a-b387-f2b99834c6a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:10:25.954460 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1061/2139] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:10:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 15:10:25.964702 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0ee759-2d65-442e-9cf2-585e563e6994 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:10:25.964987 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0ee759-2d65-442e-9cf2-585e563e6994 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:10:25.965265 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0ee759-2d65-442e-9cf2-585e563e6994 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:10:25.988154 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-7c0ee759-2d65-442e-9cf2-585e563e6994 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:10:25.989365 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-7c0ee759-2d65-442e-9cf2-585e563e6994 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:10:25.997604 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1079/2140] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:10:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 15:10:26.002195 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3466b25a-1cb7-4e6a-b387-f2b99834c6a6 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:10:26.012786 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3466b25a-1cb7-4e6a-b387-f2b99834c6a6 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:10:26.018480 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f30e258-3641-4b85-a8c3-fd10875a5284 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:10:26.019086 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f30e258-3641-4b85-a8c3-fd10875a5284 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:10:26.019372 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f30e258-3641-4b85-a8c3-fd10875a5284 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:10:26.019649 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f30e258-3641-4b85-a8c3-fd10875a5284 admin admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:10:26.020393 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f30e258-3641-4b85-a8c3-fd10875a5284 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 15:10:26.020704 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f30e258-3641-4b85-a8c3-fd10875a5284 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:10:26.022362 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f30e258-3641-4b85-a8c3-fd10875a5284 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:10:26.025143 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:10:26.037392 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1062/2141] 10.222.0.46 () {58 vars in 1152 bytes} [Thu May 26 15:10:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 15:10:26.041574 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0ee759-2d65-442e-9cf2-585e563e6994 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:10:26.051627 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0ee759-2d65-442e-9cf2-585e563e6994 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:10:26.056677 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f91dfb8-4de2-4851-91f8-bb6ea81f4890 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:10:26.057587 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f91dfb8-4de2-4851-91f8-bb6ea81f4890 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:10:26.057869 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f91dfb8-4de2-4851-91f8-bb6ea81f4890 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:10:26.058152 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f91dfb8-4de2-4851-91f8-bb6ea81f4890 admin admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:10:26.058868 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f91dfb8-4de2-4851-91f8-bb6ea81f4890 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:10:26.060776 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f91dfb8-4de2-4851-91f8-bb6ea81f4890 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:10:26.090776 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1080/2142] 10.222.0.46 () {60 vars in 1152 bytes} [Thu May 26 15:10:26 2022] POST /identity/v3/projects => generated 363 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 15:10:26.095480 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f30e258-3641-4b85-a8c3-fd10875a5284 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:10:26.106696 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f30e258-3641-4b85-a8c3-fd10875a5284 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:10:26.179996 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68683bb-5b76-47e9-b88a-9be21edc5835 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:10:26.180674 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68683bb-5b76-47e9-b88a-9be21edc5835 admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:10:26.180958 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68683bb-5b76-47e9-b88a-9be21edc5835 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:10:26.181233 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68683bb-5b76-47e9-b88a-9be21edc5835 admin admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:10:26.181993 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68683bb-5b76-47e9-b88a-9be21edc5835 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:10:26.183921 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68683bb-5b76-47e9-b88a-9be21edc5835 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:10:26.205147 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1063/2143] 10.222.0.46 () {60 vars in 1143 bytes} [Thu May 26 15:10:26 2022] POST /identity/v3/users => generated 282 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 15:10:26.209187 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f91dfb8-4de2-4851-91f8-bb6ea81f4890 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:10:26.220320 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f91dfb8-4de2-4851-91f8-bb6ea81f4890 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:10:26.224887 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6996420f-cd3d-4e41-9c7b-03522a16fc28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:10:26.225568 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6996420f-cd3d-4e41-9c7b-03522a16fc28 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:10:26.225854 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6996420f-cd3d-4e41-9c7b-03522a16fc28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:10:26.226126 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6996420f-cd3d-4e41-9c7b-03522a16fc28 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:10:26.226844 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6996420f-cd3d-4e41-9c7b-03522a16fc28 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:10:26.228431 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6996420f-cd3d-4e41-9c7b-03522a16fc28 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:10:26.238905 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1081/2144] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 15:10:26 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 15:10:26.244274 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68683bb-5b76-47e9-b88a-9be21edc5835 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:10:26.254541 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68683bb-5b76-47e9-b88a-9be21edc5835 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:10:26.259096 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f0e246-894f-49b7-8797-4666de94d15f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:10:26.259750 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f0e246-894f-49b7-8797-4666de94d15f admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:10:26.260546 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f0e246-894f-49b7-8797-4666de94d15f admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:10:26.260820 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f0e246-894f-49b7-8797-4666de94d15f admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:10:26.261570 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f0e246-894f-49b7-8797-4666de94d15f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:10:26.263156 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f0e246-894f-49b7-8797-4666de94d15f admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:10:26.273697 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1064/2145] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 15:10:26 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 15:10:26.278454 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6996420f-cd3d-4e41-9c7b-03522a16fc28 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:10:26.289274 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6996420f-cd3d-4e41-9c7b-03522a16fc28 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:10:26.294629 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9291702f-8339-4424-a755-5a749233bc1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:10:26.295660 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9291702f-8339-4424-a755-5a749233bc1b admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:10:26.296223 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9291702f-8339-4424-a755-5a749233bc1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:10:26.297061 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9291702f-8339-4424-a755-5a749233bc1b admin admin] PATH_INFO: `/v3/projects/9c26bf3f137043e09765ef925d04741d/users/3d7fab3d4427495582b0fed2b89dc06c/roles/d4088d1845a24fbd9bd8975ce0b1d3d0` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:10:26.314153 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9291702f-8339-4424-a755-5a749233bc1b admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c26bf3f137043e09765ef925d04741d, user_id=3d7fab3d4427495582b0fed2b89dc06c, role_id=d4088d1845a24fbd9bd8975ce0b1d3d0)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:10:26.316838 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9291702f-8339-4424-a755-5a749233bc1b admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:10:26.331459 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1082/2146] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 15:10:26 2022] PUT /identity/v3/projects/9c26bf3f137043e09765ef925d04741d/users/3d7fab3d4427495582b0fed2b89dc06c/roles/d4088d1845a24fbd9bd8975ce0b1d3d0 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:10:26.342441 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064bad97-f0d9-44cb-98ea-d1cc7c46c02a None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:10:26.342848 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064bad97-f0d9-44cb-98ea-d1cc7c46c02a None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:10:26.343227 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064bad97-f0d9-44cb-98ea-d1cc7c46c02a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:10:26.372186 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-064bad97-f0d9-44cb-98ea-d1cc7c46c02a None None] MFA Rules not processed for user `3d7fab3d4427495582b0fed2b89dc06c`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:10:26.373758 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-064bad97-f0d9-44cb-98ea-d1cc7c46c02a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:10:26.443842 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1065/2147] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:10:26 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:10:26.457740 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9291702f-8339-4424-a755-5a749233bc1b admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:10:26.459561 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-9291702f-8339-4424-a755-5a749233bc1b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:10:26.533046 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9291702f-8339-4424-a755-5a749233bc1b admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:10:26.537013 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927685d7-a2e8-4878-b244-5244912e48a9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5b5b35614a547d783e6e77ddb8c1c03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:10:26.537725 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927685d7-a2e8-4878-b244-5244912e48a9 service cinder] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:10:26.538217 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927685d7-a2e8-4878-b244-5244912e48a9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:10:26.538217 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-927685d7-a2e8-4878-b244-5244912e48a9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:10:26.547297 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927685d7-a2e8-4878-b244-5244912e48a9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:10:26.548848 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-927685d7-a2e8-4878-b244-5244912e48a9 service cinder] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:10:26.560677 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1083/2148] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:10:26 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 15:10:28.172350 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5431afb7-cf60-477b-b48a-219494c3ab94 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:10:28.173093 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5431afb7-cf60-477b-b48a-219494c3ab94 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:10:28.173093 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5431afb7-cf60-477b-b48a-219494c3ab94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:10:28.192026 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-5431afb7-cf60-477b-b48a-219494c3ab94 None None] MFA Rules not processed for user `1c54e915f4014d58ba28d6e6d5dd6ab6`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:10:28.193148 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-5431afb7-cf60-477b-b48a-219494c3ab94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:10:28.250187 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1066/2149] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:10:28 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:10:32.209697 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927685d7-a2e8-4878-b244-5244912e48a9 service cinder] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:10:32.212401 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-927685d7-a2e8-4878-b244-5244912e48a9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:10:32.334527 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:DELETE=3,keystone:SELECT=95,keystone:INSERT=2 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:10:32.373962 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-927685d7-a2e8-4878-b244-5244912e48a9 service cinder] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:10:32.378999 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9634848d-fb6f-407f-8de2-9f100e4f4963 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:10:32.379923 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9634848d-fb6f-407f-8de2-9f100e4f4963 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:10:32.380211 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9634848d-fb6f-407f-8de2-9f100e4f4963 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:10:32.380560 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9634848d-fb6f-407f-8de2-9f100e4f4963 admin admin] PATH_INFO: `/v3/users/88ac198b5d484006974f9f3d10728dbd` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:10:32.381753 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9634848d-fb6f-407f-8de2-9f100e4f4963 admin admin] RBAC: Authorizing `identity:delete_user(user_id=88ac198b5d484006974f9f3d10728dbd)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:10:32.383674 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9634848d-fb6f-407f-8de2-9f100e4f4963 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:10:32.439386 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-9634848d-fb6f-407f-8de2-9f100e4f4963 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '88ac198b5d484006974f9f3d10728dbd'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:10:32.447634 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-9634848d-fb6f-407f-8de2-9f100e4f4963 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '88ac198b5d484006974f9f3d10728dbd'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:10:32.448442 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-9634848d-fb6f-407f-8de2-9f100e4f4963 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '88ac198b5d484006974f9f3d10728dbd'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:10:32.465077 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-9634848d-fb6f-407f-8de2-9f100e4f4963 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '88ac198b5d484006974f9f3d10728dbd'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:10:32.479175 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-9634848d-fb6f-407f-8de2-9f100e4f4963 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '88ac198b5d484006974f9f3d10728dbd'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:10:32.485942 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1084/2150] 10.222.0.46 () {58 vars in 1223 bytes} [Thu May 26 15:10:32 2022] DELETE /identity/v3/users/88ac198b5d484006974f9f3d10728dbd => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:10:32.490766 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7a76b4-f51e-4917-bc60-8fb00d855ae2 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:10:32.491358 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7a76b4-f51e-4917-bc60-8fb00d855ae2 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:10:32.491358 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec7a76b4-f51e-4917-bc60-8fb00d855ae2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:10:32.510387 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-ec7a76b4-f51e-4917-bc60-8fb00d855ae2 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:10:32.511921 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-ec7a76b4-f51e-4917-bc60-8fb00d855ae2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:10:32.516519 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:10:32.601598 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1067/2151] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:10:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:10:32.614362 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9634848d-fb6f-407f-8de2-9f100e4f4963 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:10:32.616118 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-9634848d-fb6f-407f-8de2-9f100e4f4963 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:10:32.797424 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9634848d-fb6f-407f-8de2-9f100e4f4963 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:10:32.810417 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee164bbc-749a-40cc-bf9c-103261af1112 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0156a2f6f3794f479e76b85bdbbe5c26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:10:32.811112 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee164bbc-749a-40cc-bf9c-103261af1112 service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:10:32.811529 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee164bbc-749a-40cc-bf9c-103261af1112 service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:10:32.811823 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee164bbc-749a-40cc-bf9c-103261af1112 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:10:32.833654 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee164bbc-749a-40cc-bf9c-103261af1112 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:10:32.835583 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee164bbc-749a-40cc-bf9c-103261af1112 service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:10:32.851209 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1085/2152] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:10:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 238 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 15:10:33.348013 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7a76b4-f51e-4917-bc60-8fb00d855ae2 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:10:33.362558 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-ec7a76b4-f51e-4917-bc60-8fb00d855ae2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:10:33.406423 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec7a76b4-f51e-4917-bc60-8fb00d855ae2 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:10:33.413198 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d8aeed-7b7e-4b92-9ce7-951fec6fbd3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:10:33.414275 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d8aeed-7b7e-4b92-9ce7-951fec6fbd3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:10:33.414841 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d8aeed-7b7e-4b92-9ce7-951fec6fbd3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:10:33.415348 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d8aeed-7b7e-4b92-9ce7-951fec6fbd3d admin admin] PATH_INFO: `/v3/projects/f826f3583bc941d9a26edae3aee285ce` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:10:33.420165 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d8aeed-7b7e-4b92-9ce7-951fec6fbd3d admin admin] RBAC: Authorizing `identity:delete_project(project_id=f826f3583bc941d9a26edae3aee285ce)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:10:33.424464 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d8aeed-7b7e-4b92-9ce7-951fec6fbd3d admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:10:33.514848 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-98d8aeed-7b7e-4b92-9ce7-951fec6fbd3d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f826f3583bc941d9a26edae3aee285ce', 'request_id': 'req-98d8aeed-7b7e-4b92-9ce7-951fec6fbd3d'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:10:33.522286 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-98d8aeed-7b7e-4b92-9ce7-951fec6fbd3d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f826f3583bc941d9a26edae3aee285ce', 'request_id': 'req-98d8aeed-7b7e-4b92-9ce7-951fec6fbd3d'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:10:33.543716 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-98d8aeed-7b7e-4b92-9ce7-951fec6fbd3d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f826f3583bc941d9a26edae3aee285ce', 'request_id': 'req-98d8aeed-7b7e-4b92-9ce7-951fec6fbd3d'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:10:33.563088 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-98d8aeed-7b7e-4b92-9ce7-951fec6fbd3d admin admin] The token cache is being invalidate because project f826f3583bc941d9a26edae3aee285ce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 15:10:33.563931 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-98d8aeed-7b7e-4b92-9ce7-951fec6fbd3d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:10:33.564678 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-98d8aeed-7b7e-4b92-9ce7-951fec6fbd3d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:10:33.566681 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1068/2153] 10.222.0.46 () {58 vars in 1232 bytes} [Thu May 26 15:10:33 2022] DELETE /identity/v3/projects/f826f3583bc941d9a26edae3aee285ce => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:10:33.571844 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcd6c96-89e7-490f-8f17-a22af383164a None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:10:33.572918 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcd6c96-89e7-490f-8f17-a22af383164a None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:10:33.573137 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcd6c96-89e7-490f-8f17-a22af383164a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:10:33.592743 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-edcd6c96-89e7-490f-8f17-a22af383164a None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:10:33.594263 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-edcd6c96-89e7-490f-8f17-a22af383164a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:10:33.646691 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:10:33.671175 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1086/2154] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:10:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:10:33.679008 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061f8432-2683-4a66-84c5-38303bef3ffa None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:10:33.679314 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061f8432-2683-4a66-84c5-38303bef3ffa None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:10:33.679600 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061f8432-2683-4a66-84c5-38303bef3ffa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:10:33.699662 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-061f8432-2683-4a66-84c5-38303bef3ffa None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:10:33.700896 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-061f8432-2683-4a66-84c5-38303bef3ffa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:10:33.709414 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1069/2155] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:10:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:10:33.714001 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcd6c96-89e7-490f-8f17-a22af383164a None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:10:33.724444 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcd6c96-89e7-490f-8f17-a22af383164a None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:10:33.729763 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea37516-4ac3-45eb-8c68-25e4725bcd7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:10:33.730439 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea37516-4ac3-45eb-8c68-25e4725bcd7e admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:10:33.730727 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea37516-4ac3-45eb-8c68-25e4725bcd7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:10:33.731004 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea37516-4ac3-45eb-8c68-25e4725bcd7e admin admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:10:33.731743 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea37516-4ac3-45eb-8c68-25e4725bcd7e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 15:10:33.732055 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea37516-4ac3-45eb-8c68-25e4725bcd7e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:10:33.733715 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea37516-4ac3-45eb-8c68-25e4725bcd7e admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:10:33.746418 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1087/2156] 10.222.0.46 () {58 vars in 1152 bytes} [Thu May 26 15:10:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 15:10:33.751239 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061f8432-2683-4a66-84c5-38303bef3ffa None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:10:33.760956 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061f8432-2683-4a66-84c5-38303bef3ffa None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:10:33.766310 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30472b5-8201-498b-b6cb-f73b9ee6e64b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:10:33.766735 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30472b5-8201-498b-b6cb-f73b9ee6e64b admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:10:33.767019 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30472b5-8201-498b-b6cb-f73b9ee6e64b admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:10:33.767298 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30472b5-8201-498b-b6cb-f73b9ee6e64b admin admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:10:33.768020 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30472b5-8201-498b-b6cb-f73b9ee6e64b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:10:33.770051 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30472b5-8201-498b-b6cb-f73b9ee6e64b admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:10:33.801173 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1070/2157] 10.222.0.46 () {60 vars in 1152 bytes} [Thu May 26 15:10:33 2022] POST /identity/v3/projects => generated 373 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 15:10:33.805561 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea37516-4ac3-45eb-8c68-25e4725bcd7e admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:10:33.816811 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea37516-4ac3-45eb-8c68-25e4725bcd7e admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:10:33.878551 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511f3d7e-9786-40dd-a124-1febcc1588e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:10:33.879254 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511f3d7e-9786-40dd-a124-1febcc1588e6 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:10:33.879534 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511f3d7e-9786-40dd-a124-1febcc1588e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:10:33.879866 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511f3d7e-9786-40dd-a124-1febcc1588e6 admin admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:10:33.880571 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511f3d7e-9786-40dd-a124-1febcc1588e6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:10:33.882560 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511f3d7e-9786-40dd-a124-1febcc1588e6 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:10:33.907104 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1088/2158] 10.222.0.46 () {60 vars in 1143 bytes} [Thu May 26 15:10:33 2022] POST /identity/v3/users => generated 287 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 15:10:33.911453 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30472b5-8201-498b-b6cb-f73b9ee6e64b admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:10:33.924893 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30472b5-8201-498b-b6cb-f73b9ee6e64b admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:10:33.931662 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6614a71b-49db-406f-b044-e779e38e5f84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:10:33.934362 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6614a71b-49db-406f-b044-e779e38e5f84 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:10:33.934781 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6614a71b-49db-406f-b044-e779e38e5f84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:10:33.935165 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6614a71b-49db-406f-b044-e779e38e5f84 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:10:33.938701 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6614a71b-49db-406f-b044-e779e38e5f84 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:10:33.940495 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6614a71b-49db-406f-b044-e779e38e5f84 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:10:33.956869 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1071/2159] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 15:10:33 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 15:10:33.960445 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511f3d7e-9786-40dd-a124-1febcc1588e6 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:10:33.981310 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511f3d7e-9786-40dd-a124-1febcc1588e6 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:10:33.991212 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdca078-0070-4513-a62b-4214f134dc99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:10:33.992542 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdca078-0070-4513-a62b-4214f134dc99 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:10:33.993159 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdca078-0070-4513-a62b-4214f134dc99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:10:33.993743 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdca078-0070-4513-a62b-4214f134dc99 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:10:33.994839 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acdca078-0070-4513-a62b-4214f134dc99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:10:33.999570 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acdca078-0070-4513-a62b-4214f134dc99 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:10:34.013369 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1089/2160] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 15:10:33 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 15:10:34.020039 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6614a71b-49db-406f-b044-e779e38e5f84 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:10:34.031970 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6614a71b-49db-406f-b044-e779e38e5f84 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:10:34.037077 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b92b508-01ec-4f24-bb7d-9a0f0e231a10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:10:34.038046 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b92b508-01ec-4f24-bb7d-9a0f0e231a10 admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:10:34.038328 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b92b508-01ec-4f24-bb7d-9a0f0e231a10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:10:34.038607 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b92b508-01ec-4f24-bb7d-9a0f0e231a10 admin admin] PATH_INFO: `/v3/projects/a272160dccba4f6b870efe8e25dae635/users/10a511609a9a4875845a1056be541667/roles/d4088d1845a24fbd9bd8975ce0b1d3d0` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:10:34.065498 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b92b508-01ec-4f24-bb7d-9a0f0e231a10 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a272160dccba4f6b870efe8e25dae635, user_id=10a511609a9a4875845a1056be541667, role_id=d4088d1845a24fbd9bd8975ce0b1d3d0)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:10:34.065498 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b92b508-01ec-4f24-bb7d-9a0f0e231a10 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:10:34.078459 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1072/2161] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 15:10:34 2022] PUT /identity/v3/projects/a272160dccba4f6b870efe8e25dae635/users/10a511609a9a4875845a1056be541667/roles/d4088d1845a24fbd9bd8975ce0b1d3d0 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:10:34.090327 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312c27d2-af02-4c55-8d88-e403d0992787 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:10:34.090612 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312c27d2-af02-4c55-8d88-e403d0992787 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:10:34.090914 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312c27d2-af02-4c55-8d88-e403d0992787 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:10:34.124597 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-312c27d2-af02-4c55-8d88-e403d0992787 None None] MFA Rules not processed for user `10a511609a9a4875845a1056be541667`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:10:34.125921 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-312c27d2-af02-4c55-8d88-e403d0992787 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:10:34.221934 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1090/2162] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:10:34 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 15:10:34.233745 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b92b508-01ec-4f24-bb7d-9a0f0e231a10 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:10:34.235477 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-5b92b508-01ec-4f24-bb7d-9a0f0e231a10 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:10:34.273317 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:10:34.319631 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b92b508-01ec-4f24-bb7d-9a0f0e231a10 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:10:34.325430 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86e67ee-ca8a-45b4-b6f0-810338f6f2f8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5b5b35614a547d783e6e77ddb8c1c03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:10:34.326375 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86e67ee-ca8a-45b4-b6f0-810338f6f2f8 service cinder] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:10:34.326733 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86e67ee-ca8a-45b4-b6f0-810338f6f2f8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:10:34.327083 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86e67ee-ca8a-45b4-b6f0-810338f6f2f8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:10:34.343544 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86e67ee-ca8a-45b4-b6f0-810338f6f2f8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:10:34.345510 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86e67ee-ca8a-45b4-b6f0-810338f6f2f8 service cinder] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:10:34.357784 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1073/2163] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:10:34 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 15:10:36.045616 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3308b7e6-5897-4da3-9199-203c297822c1 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:10:36.046114 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3308b7e6-5897-4da3-9199-203c297822c1 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:10:36.046192 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3308b7e6-5897-4da3-9199-203c297822c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:10:36.069929 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-3308b7e6-5897-4da3-9199-203c297822c1 None None] MFA Rules not processed for user `10a511609a9a4875845a1056be541667`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:10:36.071220 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-3308b7e6-5897-4da3-9199-203c297822c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:10:36.082792 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1091/2164] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:10:36 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:10:40.471599 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86e67ee-ca8a-45b4-b6f0-810338f6f2f8 service cinder] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:10:40.472309 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-a86e67ee-ca8a-45b4-b6f0-810338f6f2f8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:10:40.561624 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86e67ee-ca8a-45b4-b6f0-810338f6f2f8 service cinder] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:10:40.568119 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c648336-a66d-43c4-8c5e-b6b8376716a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:10:40.569403 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c648336-a66d-43c4-8c5e-b6b8376716a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:10:40.569732 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c648336-a66d-43c4-8c5e-b6b8376716a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:10:40.570365 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c648336-a66d-43c4-8c5e-b6b8376716a1 admin admin] PATH_INFO: `/v3/users/7ee3e4c99a6347fc8d95ecda7e74b4ad` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:10:40.572013 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c648336-a66d-43c4-8c5e-b6b8376716a1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7ee3e4c99a6347fc8d95ecda7e74b4ad)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:10:40.574767 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c648336-a66d-43c4-8c5e-b6b8376716a1 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:10:40.631831 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-4c648336-a66d-43c4-8c5e-b6b8376716a1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7ee3e4c99a6347fc8d95ecda7e74b4ad'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:10:40.639540 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-4c648336-a66d-43c4-8c5e-b6b8376716a1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7ee3e4c99a6347fc8d95ecda7e74b4ad'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:10:40.640193 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-4c648336-a66d-43c4-8c5e-b6b8376716a1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7ee3e4c99a6347fc8d95ecda7e74b4ad'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:10:40.654157 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-4c648336-a66d-43c4-8c5e-b6b8376716a1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7ee3e4c99a6347fc8d95ecda7e74b4ad'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:10:40.663880 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-4c648336-a66d-43c4-8c5e-b6b8376716a1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7ee3e4c99a6347fc8d95ecda7e74b4ad'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:10:40.674934 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1074/2165] 10.222.0.46 () {58 vars in 1223 bytes} [Thu May 26 15:10:40 2022] DELETE /identity/v3/users/7ee3e4c99a6347fc8d95ecda7e74b4ad => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:10:40.676646 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6371ab9-17bc-4025-bbf3-b34828f44b41 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:10:40.677110 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6371ab9-17bc-4025-bbf3-b34828f44b41 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:10:40.677188 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6371ab9-17bc-4025-bbf3-b34828f44b41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:10:40.740560 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-d6371ab9-17bc-4025-bbf3-b34828f44b41 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:10:40.743452 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-d6371ab9-17bc-4025-bbf3-b34828f44b41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:10:40.778680 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:10:40.831165 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1092/2166] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:10:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:10:40.841355 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c648336-a66d-43c4-8c5e-b6b8376716a1 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:10:40.842962 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-4c648336-a66d-43c4-8c5e-b6b8376716a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:10:40.888108 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:10:40.964537 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c648336-a66d-43c4-8c5e-b6b8376716a1 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:10:40.968568 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac5ef51-8ee8-483e-ac62-64376f4a3af2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0156a2f6f3794f479e76b85bdbbe5c26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:10:40.969248 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac5ef51-8ee8-483e-ac62-64376f4a3af2 service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:10:40.969559 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac5ef51-8ee8-483e-ac62-64376f4a3af2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:10:40.970142 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac5ef51-8ee8-483e-ac62-64376f4a3af2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:10:40.980759 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac5ef51-8ee8-483e-ac62-64376f4a3af2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:10:40.982252 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac5ef51-8ee8-483e-ac62-64376f4a3af2 service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:10:40.993152 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1075/2167] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:10:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 15:10:41.357355 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6371ab9-17bc-4025-bbf3-b34828f44b41 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:10:41.358918 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-d6371ab9-17bc-4025-bbf3-b34828f44b41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:10:41.377788 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6371ab9-17bc-4025-bbf3-b34828f44b41 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:10:41.384009 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc879f6-1b6d-44bf-b330-7032160f4ce3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:10:41.386084 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc879f6-1b6d-44bf-b330-7032160f4ce3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:10:41.386378 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc879f6-1b6d-44bf-b330-7032160f4ce3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:10:41.386729 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc879f6-1b6d-44bf-b330-7032160f4ce3 admin admin] PATH_INFO: `/v3/projects/1dc3721222594b3fa82bdf972888592f` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:10:41.388159 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc879f6-1b6d-44bf-b330-7032160f4ce3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1dc3721222594b3fa82bdf972888592f)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:10:41.390232 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc879f6-1b6d-44bf-b330-7032160f4ce3 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:10:41.447889 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-bdc879f6-1b6d-44bf-b330-7032160f4ce3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1dc3721222594b3fa82bdf972888592f', 'request_id': 'req-bdc879f6-1b6d-44bf-b330-7032160f4ce3'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:10:41.456363 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-bdc879f6-1b6d-44bf-b330-7032160f4ce3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1dc3721222594b3fa82bdf972888592f', 'request_id': 'req-bdc879f6-1b6d-44bf-b330-7032160f4ce3'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:10:41.462093 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-bdc879f6-1b6d-44bf-b330-7032160f4ce3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1dc3721222594b3fa82bdf972888592f', 'request_id': 'req-bdc879f6-1b6d-44bf-b330-7032160f4ce3'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:10:41.473698 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-bdc879f6-1b6d-44bf-b330-7032160f4ce3 admin admin] The token cache is being invalidate because project 1dc3721222594b3fa82bdf972888592f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 15:10:41.474004 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-bdc879f6-1b6d-44bf-b330-7032160f4ce3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:10:41.474635 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-bdc879f6-1b6d-44bf-b330-7032160f4ce3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:10:41.476178 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1093/2168] 10.222.0.46 () {58 vars in 1232 bytes} [Thu May 26 15:10:41 2022] DELETE /identity/v3/projects/1dc3721222594b3fa82bdf972888592f => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:10:50.988872 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=20 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:10:51.470518 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=43,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:10:58.722396 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc8d13e-e63a-4ecb-abde-adfb54c5663b None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:10:58.722923 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc8d13e-e63a-4ecb-abde-adfb54c5663b None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:10:58.723096 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc8d13e-e63a-4ecb-abde-adfb54c5663b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:10:58.753911 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-8cc8d13e-e63a-4ecb-abde-adfb54c5663b None None] MFA Rules not processed for user `8028c520d93449a6a087c281280ee13d`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:10:58.755039 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-8cc8d13e-e63a-4ecb-abde-adfb54c5663b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:10:58.833145 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1076/2169] 10.222.0.46 () {58 vars in 1009 bytes} [Thu May 26 15:10:58 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:10:58.890641 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc879f6-1b6d-44bf-b330-7032160f4ce3 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:10:58.892189 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-bdc879f6-1b6d-44bf-b330-7032160f4ce3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:10:59.055139 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc879f6-1b6d-44bf-b330-7032160f4ce3 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:10:59.062710 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0314e9c-1a1f-4988-b9ab-adbb7bd5abe7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4bea272624b45d8ac2ad27bd07df885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:10:59.063999 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0314e9c-1a1f-4988-b9ab-adbb7bd5abe7 service swift] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:10:59.064466 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0314e9c-1a1f-4988-b9ab-adbb7bd5abe7 service swift] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:10:59.065077 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0314e9c-1a1f-4988-b9ab-adbb7bd5abe7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:10:59.076097 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0314e9c-1a1f-4988-b9ab-adbb7bd5abe7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:10:59.078816 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0314e9c-1a1f-4988-b9ab-adbb7bd5abe7 service swift] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:10:59.109714 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1094/2170] 10.222.0.46 () {60 vars in 1384 bytes} [Thu May 26 15:10:58 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 26 15:11:00.658127 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8aef4d-043a-4e58-8166-5799068ed2fb None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:11:00.659092 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8aef4d-043a-4e58-8166-5799068ed2fb None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:11:00.659530 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8aef4d-043a-4e58-8166-5799068ed2fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:11:00.681403 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-0c8aef4d-043a-4e58-8166-5799068ed2fb None None] MFA Rules not processed for user `8028c520d93449a6a087c281280ee13d`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:11:00.682947 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-0c8aef4d-043a-4e58-8166-5799068ed2fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:11:00.699524 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1077/2171] 10.222.0.46 () {58 vars in 1009 bytes} [Thu May 26 15:11:00 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:11:00.717799 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0314e9c-1a1f-4988-b9ab-adbb7bd5abe7 service swift] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:11:00.732669 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0314e9c-1a1f-4988-b9ab-adbb7bd5abe7 service swift] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:11:00.738664 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9614f6-2e98-4418-918b-5d7219d01eca service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4bea272624b45d8ac2ad27bd07df885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:11:00.739581 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9614f6-2e98-4418-918b-5d7219d01eca service swift] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:11:00.740037 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9614f6-2e98-4418-918b-5d7219d01eca service swift] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:11:00.740521 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9614f6-2e98-4418-918b-5d7219d01eca service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:11:00.753060 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9614f6-2e98-4418-918b-5d7219d01eca service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:11:00.754800 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9614f6-2e98-4418-918b-5d7219d01eca service swift] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:11:00.768452 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1095/2172] 10.222.0.46 () {60 vars in 1384 bytes} [Thu May 26 15:11:00 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 26 15:11:01.569619 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09570894-b39d-4947-8b2b-74529503d04e None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:11:01.570158 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09570894-b39d-4947-8b2b-74529503d04e None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:11:01.570585 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09570894-b39d-4947-8b2b-74529503d04e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:11:01.598445 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-09570894-b39d-4947-8b2b-74529503d04e None None] MFA Rules not processed for user `8028c520d93449a6a087c281280ee13d`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:11:01.599714 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-09570894-b39d-4947-8b2b-74529503d04e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:11:01.608922 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1078/2173] 10.222.0.46 () {58 vars in 967 bytes} [Thu May 26 15:11:01 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:11:01.628280 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9614f6-2e98-4418-918b-5d7219d01eca service swift] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:11:01.654665 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9614f6-2e98-4418-918b-5d7219d01eca service swift] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:11:01.669604 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61588e41-2fea-458c-96b8-ef2af17f5e28 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4bea272624b45d8ac2ad27bd07df885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:11:01.670583 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61588e41-2fea-458c-96b8-ef2af17f5e28 service swift] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:11:01.671093 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61588e41-2fea-458c-96b8-ef2af17f5e28 service swift] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:11:01.671379 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61588e41-2fea-458c-96b8-ef2af17f5e28 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:11:01.720332 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61588e41-2fea-458c-96b8-ef2af17f5e28 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:11:01.722689 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61588e41-2fea-458c-96b8-ef2af17f5e28 service swift] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:11:01.734109 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1096/2174] 10.222.0.46 () {60 vars in 1384 bytes} [Thu May 26 15:11:01 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 26 15:11:02.029073 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bef9c38-880f-4cde-9450-af775a11fa02 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:11:02.029698 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bef9c38-880f-4cde-9450-af775a11fa02 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:11:02.030305 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bef9c38-880f-4cde-9450-af775a11fa02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:11:02.052404 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-9bef9c38-880f-4cde-9450-af775a11fa02 None None] MFA Rules not processed for user `8028c520d93449a6a087c281280ee13d`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:11:02.054151 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-9bef9c38-880f-4cde-9450-af775a11fa02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:11:02.068986 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1079/2175] 10.222.0.46 () {58 vars in 1009 bytes} [Thu May 26 15:11:02 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 15:11:02.078773 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61588e41-2fea-458c-96b8-ef2af17f5e28 service swift] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:11:02.089948 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61588e41-2fea-458c-96b8-ef2af17f5e28 service swift] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:11:02.100879 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83fc4f5-91a1-46ca-a867-e4fbff8db808 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4bea272624b45d8ac2ad27bd07df885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:11:02.101695 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83fc4f5-91a1-46ca-a867-e4fbff8db808 service swift] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:11:02.102085 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83fc4f5-91a1-46ca-a867-e4fbff8db808 service swift] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:11:02.102371 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83fc4f5-91a1-46ca-a867-e4fbff8db808 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:11:02.114061 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83fc4f5-91a1-46ca-a867-e4fbff8db808 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:11:02.116002 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83fc4f5-91a1-46ca-a867-e4fbff8db808 service swift] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:11:02.128323 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1097/2176] 10.222.0.46 () {60 vars in 1384 bytes} [Thu May 26 15:11:02 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 26 15:11:02.647294 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4b75cf-327f-418d-90ba-b256c50e3d1f None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:11:02.648665 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4b75cf-327f-418d-90ba-b256c50e3d1f None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:11:02.649110 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4b75cf-327f-418d-90ba-b256c50e3d1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:11:02.670496 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-4c4b75cf-327f-418d-90ba-b256c50e3d1f None None] MFA Rules not processed for user `8028c520d93449a6a087c281280ee13d`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:11:02.672217 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-4c4b75cf-327f-418d-90ba-b256c50e3d1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:11:02.691588 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1080/2177] 10.222.0.46 () {58 vars in 967 bytes} [Thu May 26 15:11:02 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:11:02.702525 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83fc4f5-91a1-46ca-a867-e4fbff8db808 service swift] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:11:02.715369 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83fc4f5-91a1-46ca-a867-e4fbff8db808 service swift] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:11:02.725506 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa835378-5c5c-4bbe-bf41-9bd2fe1e726f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4bea272624b45d8ac2ad27bd07df885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:11:02.726279 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa835378-5c5c-4bbe-bf41-9bd2fe1e726f service swift] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:11:02.726589 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa835378-5c5c-4bbe-bf41-9bd2fe1e726f service swift] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:11:02.726914 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa835378-5c5c-4bbe-bf41-9bd2fe1e726f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:11:02.740693 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa835378-5c5c-4bbe-bf41-9bd2fe1e726f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:11:02.743582 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa835378-5c5c-4bbe-bf41-9bd2fe1e726f service swift] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:11:02.755791 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1098/2178] 10.222.0.46 () {60 vars in 1384 bytes} [Thu May 26 15:11:02 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 26 15:11:05.028677 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a5a14d-39d5-4393-a31f-9a35c5c8bcdf None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:11:05.029794 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a5a14d-39d5-4393-a31f-9a35c5c8bcdf None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:11:05.030323 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a5a14d-39d5-4393-a31f-9a35c5c8bcdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:11:05.063831 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-e0a5a14d-39d5-4393-a31f-9a35c5c8bcdf None None] MFA Rules not processed for user `8028c520d93449a6a087c281280ee13d`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:11:05.065599 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-e0a5a14d-39d5-4393-a31f-9a35c5c8bcdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:11:05.080545 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1081/2179] 10.222.0.46 () {58 vars in 1009 bytes} [Thu May 26 15:11:05 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:11:05.123459 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa835378-5c5c-4bbe-bf41-9bd2fe1e726f service swift] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:11:05.149708 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa835378-5c5c-4bbe-bf41-9bd2fe1e726f service swift] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:11:05.151176 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5015b97e-e567-48f9-a8ac-886444308837 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4bea272624b45d8ac2ad27bd07df885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:11:05.151833 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5015b97e-e567-48f9-a8ac-886444308837 service swift] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:11:05.152117 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5015b97e-e567-48f9-a8ac-886444308837 service swift] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:11:05.152391 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5015b97e-e567-48f9-a8ac-886444308837 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:11:05.166356 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5015b97e-e567-48f9-a8ac-886444308837 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:11:05.168800 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5015b97e-e567-48f9-a8ac-886444308837 service swift] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:11:05.179477 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1099/2180] 10.222.0.46 () {60 vars in 1384 bytes} [Thu May 26 15:11:05 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 26 15:11:06.576519 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a5a14d-39d5-4393-a31f-9a35c5c8bcdf None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:11:06.578529 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-e0a5a14d-39d5-4393-a31f-9a35c5c8bcdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:11:06.780869 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a5a14d-39d5-4393-a31f-9a35c5c8bcdf None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:11:06.796482 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac87bc2f-addf-4c8b-a4c1-a929bf7943bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:11:06.799259 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac87bc2f-addf-4c8b-a4c1-a929bf7943bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:11:06.799750 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac87bc2f-addf-4c8b-a4c1-a929bf7943bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:11:06.801183 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac87bc2f-addf-4c8b-a4c1-a929bf7943bf admin admin] PATH_INFO: `/v3/users/10a511609a9a4875845a1056be541667` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:11:06.803850 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac87bc2f-addf-4c8b-a4c1-a929bf7943bf admin admin] RBAC: Authorizing `identity:delete_user(user_id=10a511609a9a4875845a1056be541667)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:11:06.807045 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac87bc2f-addf-4c8b-a4c1-a929bf7943bf admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:11:06.917617 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-ac87bc2f-addf-4c8b-a4c1-a929bf7943bf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '10a511609a9a4875845a1056be541667'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:11:06.930958 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-ac87bc2f-addf-4c8b-a4c1-a929bf7943bf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '10a511609a9a4875845a1056be541667'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:11:06.932277 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-ac87bc2f-addf-4c8b-a4c1-a929bf7943bf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '10a511609a9a4875845a1056be541667'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:11:06.945802 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:11:06.993054 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-ac87bc2f-addf-4c8b-a4c1-a929bf7943bf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '10a511609a9a4875845a1056be541667'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:11:07.014344 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-ac87bc2f-addf-4c8b-a4c1-a929bf7943bf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '10a511609a9a4875845a1056be541667'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:11:07.024952 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1082/2181] 10.222.0.46 () {58 vars in 1223 bytes} [Thu May 26 15:11:06 2022] DELETE /identity/v3/users/10a511609a9a4875845a1056be541667 => generated 0 bytes in 449 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:11:07.037197 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4780f9-8485-4738-92eb-3c09c71a603c None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:11:07.041763 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4780f9-8485-4738-92eb-3c09c71a603c None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:11:07.043285 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4780f9-8485-4738-92eb-3c09c71a603c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:11:07.094899 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-8e4780f9-8485-4738-92eb-3c09c71a603c None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:11:07.096157 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-8e4780f9-8485-4738-92eb-3c09c71a603c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:11:07.228020 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1100/2182] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:11:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:11:07.239878 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac87bc2f-addf-4c8b-a4c1-a929bf7943bf admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:11:07.241359 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-ac87bc2f-addf-4c8b-a4c1-a929bf7943bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:11:07.364351 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac87bc2f-addf-4c8b-a4c1-a929bf7943bf admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:11:07.370046 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e58c31-5fb8-40d4-9778-e7fb40e60f65 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0156a2f6f3794f479e76b85bdbbe5c26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:11:07.370718 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e58c31-5fb8-40d4-9778-e7fb40e60f65 service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:11:07.371006 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e58c31-5fb8-40d4-9778-e7fb40e60f65 service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:11:07.371285 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e58c31-5fb8-40d4-9778-e7fb40e60f65 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:11:07.389842 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e58c31-5fb8-40d4-9778-e7fb40e60f65 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:11:07.391560 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e58c31-5fb8-40d4-9778-e7fb40e60f65 service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:11:07.437600 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1083/2183] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:11:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 15:11:07.821703 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4780f9-8485-4738-92eb-3c09c71a603c None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:11:07.823512 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-8e4780f9-8485-4738-92eb-3c09c71a603c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:11:07.848686 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4780f9-8485-4738-92eb-3c09c71a603c None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:11:07.871951 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9265bcad-4e9c-4f5c-a053-b6c1458941f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:11:07.873561 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9265bcad-4e9c-4f5c-a053-b6c1458941f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:11:07.878029 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9265bcad-4e9c-4f5c-a053-b6c1458941f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:11:07.878613 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9265bcad-4e9c-4f5c-a053-b6c1458941f4 admin admin] PATH_INFO: `/v3/projects/a272160dccba4f6b870efe8e25dae635` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:11:07.880051 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9265bcad-4e9c-4f5c-a053-b6c1458941f4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a272160dccba4f6b870efe8e25dae635)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:11:07.882241 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9265bcad-4e9c-4f5c-a053-b6c1458941f4 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:11:07.926718 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:11:07.982168 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-9265bcad-4e9c-4f5c-a053-b6c1458941f4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a272160dccba4f6b870efe8e25dae635', 'request_id': 'req-9265bcad-4e9c-4f5c-a053-b6c1458941f4'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:11:07.999927 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-9265bcad-4e9c-4f5c-a053-b6c1458941f4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a272160dccba4f6b870efe8e25dae635', 'request_id': 'req-9265bcad-4e9c-4f5c-a053-b6c1458941f4'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:11:08.012504 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-9265bcad-4e9c-4f5c-a053-b6c1458941f4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a272160dccba4f6b870efe8e25dae635', 'request_id': 'req-9265bcad-4e9c-4f5c-a053-b6c1458941f4'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:11:08.029262 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-9265bcad-4e9c-4f5c-a053-b6c1458941f4 admin admin] The token cache is being invalidate because project a272160dccba4f6b870efe8e25dae635 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 15:11:08.029676 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-9265bcad-4e9c-4f5c-a053-b6c1458941f4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:11:08.030897 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-9265bcad-4e9c-4f5c-a053-b6c1458941f4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:11:08.033148 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1101/2184] 10.222.0.46 () {58 vars in 1232 bytes} [Thu May 26 15:11:07 2022] DELETE /identity/v3/projects/a272160dccba4f6b870efe8e25dae635 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:11:15.101772 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e58c31-5fb8-40d4-9778-e7fb40e60f65 service neutron] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:11:15.104103 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-d4e58c31-5fb8-40d4-9778-e7fb40e60f65 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:11:15.119362 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e58c31-5fb8-40d4-9778-e7fb40e60f65 service neutron] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:11:15.207978 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0235acef-d017-42d7-851f-adae82de29b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:11:15.209590 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0235acef-d017-42d7-851f-adae82de29b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:11:15.210337 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0235acef-d017-42d7-851f-adae82de29b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:11:15.213381 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0235acef-d017-42d7-851f-adae82de29b1 admin admin] PATH_INFO: `/v3/users/3d7fab3d4427495582b0fed2b89dc06c` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:11:15.215784 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0235acef-d017-42d7-851f-adae82de29b1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3d7fab3d4427495582b0fed2b89dc06c)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:11:15.219790 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0235acef-d017-42d7-851f-adae82de29b1 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:11:15.294667 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-0235acef-d017-42d7-851f-adae82de29b1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3d7fab3d4427495582b0fed2b89dc06c'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:11:15.311839 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-0235acef-d017-42d7-851f-adae82de29b1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3d7fab3d4427495582b0fed2b89dc06c'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:11:15.312712 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-0235acef-d017-42d7-851f-adae82de29b1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3d7fab3d4427495582b0fed2b89dc06c'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:11:15.337944 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-0235acef-d017-42d7-851f-adae82de29b1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3d7fab3d4427495582b0fed2b89dc06c'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:11:15.349421 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-0235acef-d017-42d7-851f-adae82de29b1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3d7fab3d4427495582b0fed2b89dc06c'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:11:15.351170 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:11:15.362331 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bbf50b-dbef-42be-989f-97a9193be5d0 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:11:15.363483 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bbf50b-dbef-42be-989f-97a9193be5d0 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:11:15.363483 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bbf50b-dbef-42be-989f-97a9193be5d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:11:15.364543 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1084/2185] 10.222.0.46 () {58 vars in 1223 bytes} [Thu May 26 15:11:15 2022] DELETE /identity/v3/users/3d7fab3d4427495582b0fed2b89dc06c => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:11:15.386363 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-31bbf50b-dbef-42be-989f-97a9193be5d0 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:11:15.387864 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-31bbf50b-dbef-42be-989f-97a9193be5d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:11:15.469581 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1102/2186] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:11:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:11:15.479508 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0235acef-d017-42d7-851f-adae82de29b1 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:11:15.481895 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-0235acef-d017-42d7-851f-adae82de29b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:11:15.559151 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0235acef-d017-42d7-851f-adae82de29b1 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:11:15.569215 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0228fe2-8ed5-418f-83c0-0a45a9e18cd4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0156a2f6f3794f479e76b85bdbbe5c26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:11:15.570334 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0228fe2-8ed5-418f-83c0-0a45a9e18cd4 service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:11:15.570899 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0228fe2-8ed5-418f-83c0-0a45a9e18cd4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:11:15.571541 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0228fe2-8ed5-418f-83c0-0a45a9e18cd4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:11:15.589394 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0228fe2-8ed5-418f-83c0-0a45a9e18cd4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:11:15.591952 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0228fe2-8ed5-418f-83c0-0a45a9e18cd4 service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:11:15.612275 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1085/2187] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:11:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 15:11:15.657208 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d32a5db-12a8-48b2-8040-d64384cc04c9 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:11:15.657547 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d32a5db-12a8-48b2-8040-d64384cc04c9 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:11:15.657870 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d32a5db-12a8-48b2-8040-d64384cc04c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:11:15.684736 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-9d32a5db-12a8-48b2-8040-d64384cc04c9 None None] MFA Rules not processed for user `2ce5b5d5a12743bc80a4b191c24ff52f`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:11:15.686661 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-9d32a5db-12a8-48b2-8040-d64384cc04c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:11:15.776707 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1103/2188] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:11:15 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 15:11:16.286915 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0228fe2-8ed5-418f-83c0-0a45a9e18cd4 service neutron] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:11:16.288447 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-e0228fe2-8ed5-418f-83c0-0a45a9e18cd4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:11:16.306980 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0228fe2-8ed5-418f-83c0-0a45a9e18cd4 service neutron] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:11:16.316619 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61474d7c-5595-4d7e-9ba7-a7f1aab8d377 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:11:16.317719 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61474d7c-5595-4d7e-9ba7-a7f1aab8d377 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:11:16.318005 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61474d7c-5595-4d7e-9ba7-a7f1aab8d377 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:11:16.318849 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61474d7c-5595-4d7e-9ba7-a7f1aab8d377 admin admin] PATH_INFO: `/v3/projects/9c26bf3f137043e09765ef925d04741d` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:11:16.319882 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61474d7c-5595-4d7e-9ba7-a7f1aab8d377 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9c26bf3f137043e09765ef925d04741d)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:11:16.322511 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61474d7c-5595-4d7e-9ba7-a7f1aab8d377 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:11:16.410104 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-61474d7c-5595-4d7e-9ba7-a7f1aab8d377 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9c26bf3f137043e09765ef925d04741d', 'request_id': 'req-61474d7c-5595-4d7e-9ba7-a7f1aab8d377'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:11:16.420726 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-61474d7c-5595-4d7e-9ba7-a7f1aab8d377 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9c26bf3f137043e09765ef925d04741d', 'request_id': 'req-61474d7c-5595-4d7e-9ba7-a7f1aab8d377'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:11:16.427883 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-61474d7c-5595-4d7e-9ba7-a7f1aab8d377 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9c26bf3f137043e09765ef925d04741d', 'request_id': 'req-61474d7c-5595-4d7e-9ba7-a7f1aab8d377'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:11:16.459450 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-61474d7c-5595-4d7e-9ba7-a7f1aab8d377 admin admin] The token cache is being invalidate because project 9c26bf3f137043e09765ef925d04741d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 15:11:16.460077 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-61474d7c-5595-4d7e-9ba7-a7f1aab8d377 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:11:16.462109 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-61474d7c-5595-4d7e-9ba7-a7f1aab8d377 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:11:16.464835 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1086/2189] 10.222.0.46 () {58 vars in 1232 bytes} [Thu May 26 15:11:16 2022] DELETE /identity/v3/projects/9c26bf3f137043e09765ef925d04741d => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:11:16.471108 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893511a8-99bc-48f2-a488-f327377fa765 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:11:16.472114 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893511a8-99bc-48f2-a488-f327377fa765 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:11:16.472963 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893511a8-99bc-48f2-a488-f327377fa765 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:11:16.556496 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-893511a8-99bc-48f2-a488-f327377fa765 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:11:16.556496 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-893511a8-99bc-48f2-a488-f327377fa765 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:11:16.598400 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:11:16.683838 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1104/2190] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:11:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 15:11:16.692095 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1346b3-a8ac-4d75-9bd1-3456f580ab99 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:11:16.692426 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1346b3-a8ac-4d75-9bd1-3456f580ab99 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:11:16.692664 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1346b3-a8ac-4d75-9bd1-3456f580ab99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:11:16.716072 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-af1346b3-a8ac-4d75-9bd1-3456f580ab99 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:11:16.717312 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-af1346b3-a8ac-4d75-9bd1-3456f580ab99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:11:16.732852 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893511a8-99bc-48f2-a488-f327377fa765 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:11:16.733412 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1087/2191] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:11:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:11:16.743357 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893511a8-99bc-48f2-a488-f327377fa765 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:11:16.749269 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec837f8-b3ec-4db3-9d83-bc06b4067916 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:11:16.749682 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec837f8-b3ec-4db3-9d83-bc06b4067916 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:11:16.749968 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec837f8-b3ec-4db3-9d83-bc06b4067916 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:11:16.750413 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec837f8-b3ec-4db3-9d83-bc06b4067916 admin admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:11:16.751144 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec837f8-b3ec-4db3-9d83-bc06b4067916 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 15:11:16.752095 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec837f8-b3ec-4db3-9d83-bc06b4067916 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:11:16.753777 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec837f8-b3ec-4db3-9d83-bc06b4067916 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:11:16.772171 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1105/2192] 10.222.0.46 () {58 vars in 1152 bytes} [Thu May 26 15:11:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 15:11:16.778766 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1346b3-a8ac-4d75-9bd1-3456f580ab99 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:11:16.793658 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1346b3-a8ac-4d75-9bd1-3456f580ab99 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:11:16.802656 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf2490b-edae-4780-a8da-b62b5ba253cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:11:16.803483 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf2490b-edae-4780-a8da-b62b5ba253cc admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:11:16.803829 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf2490b-edae-4780-a8da-b62b5ba253cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:11:16.804633 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf2490b-edae-4780-a8da-b62b5ba253cc admin admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:11:16.805887 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf2490b-edae-4780-a8da-b62b5ba253cc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:11:16.809434 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf2490b-edae-4780-a8da-b62b5ba253cc admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:11:16.842393 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1088/2193] 10.222.0.46 () {60 vars in 1152 bytes} [Thu May 26 15:11:16 2022] POST /identity/v3/projects => generated 381 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 15:11:16.846593 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec837f8-b3ec-4db3-9d83-bc06b4067916 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:11:16.863465 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec837f8-b3ec-4db3-9d83-bc06b4067916 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:11:16.952957 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b50584a-abe3-4b24-88e1-7e64ac133039 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:11:16.953925 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b50584a-abe3-4b24-88e1-7e64ac133039 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:11:16.954339 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b50584a-abe3-4b24-88e1-7e64ac133039 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:11:16.954701 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b50584a-abe3-4b24-88e1-7e64ac133039 admin admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:11:16.955509 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b50584a-abe3-4b24-88e1-7e64ac133039 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:11:16.957775 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b50584a-abe3-4b24-88e1-7e64ac133039 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:11:16.990143 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1106/2194] 10.222.0.46 () {60 vars in 1143 bytes} [Thu May 26 15:11:16 2022] POST /identity/v3/users => generated 291 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 15:11:16.994054 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf2490b-edae-4780-a8da-b62b5ba253cc admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:11:17.010816 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf2490b-edae-4780-a8da-b62b5ba253cc admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:11:17.030325 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf0b7a5-2b97-4dc7-b7e1-6a4d23cdb01f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:11:17.031438 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf0b7a5-2b97-4dc7-b7e1-6a4d23cdb01f admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:11:17.031438 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf0b7a5-2b97-4dc7-b7e1-6a4d23cdb01f admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:11:17.031609 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf0b7a5-2b97-4dc7-b7e1-6a4d23cdb01f admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:11:17.032276 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf0b7a5-2b97-4dc7-b7e1-6a4d23cdb01f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:11:17.034442 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf0b7a5-2b97-4dc7-b7e1-6a4d23cdb01f admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:11:17.056460 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1089/2195] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 15:11:16 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 15:11:17.060932 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b50584a-abe3-4b24-88e1-7e64ac133039 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:11:17.075355 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b50584a-abe3-4b24-88e1-7e64ac133039 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:11:17.086431 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff6c083-679b-48ab-8f2f-2d2b5ddaa6eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:11:17.087938 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff6c083-679b-48ab-8f2f-2d2b5ddaa6eb admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:11:17.088354 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff6c083-679b-48ab-8f2f-2d2b5ddaa6eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:11:17.089247 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff6c083-679b-48ab-8f2f-2d2b5ddaa6eb admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:11:17.090211 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff6c083-679b-48ab-8f2f-2d2b5ddaa6eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:11:17.094354 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff6c083-679b-48ab-8f2f-2d2b5ddaa6eb admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:11:17.106612 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1107/2196] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 15:11:17 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 15:11:17.112489 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf0b7a5-2b97-4dc7-b7e1-6a4d23cdb01f admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:11:17.127272 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf0b7a5-2b97-4dc7-b7e1-6a4d23cdb01f admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:11:17.140355 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f469a8-1a32-46a7-8951-8310d70e6c06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:11:17.141542 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f469a8-1a32-46a7-8951-8310d70e6c06 admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:11:17.142038 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f469a8-1a32-46a7-8951-8310d70e6c06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:11:17.142425 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f469a8-1a32-46a7-8951-8310d70e6c06 admin admin] PATH_INFO: `/v3/projects/d58410e922bd46418c9e93bf61b348f8/users/dfa425e42202421c9e1c8564456c67f7/roles/d4088d1845a24fbd9bd8975ce0b1d3d0` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:11:17.200076 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f469a8-1a32-46a7-8951-8310d70e6c06 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d58410e922bd46418c9e93bf61b348f8, user_id=dfa425e42202421c9e1c8564456c67f7, role_id=d4088d1845a24fbd9bd8975ce0b1d3d0)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:11:17.211854 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f469a8-1a32-46a7-8951-8310d70e6c06 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:11:17.238093 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1090/2197] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 15:11:17 2022] PUT /identity/v3/projects/d58410e922bd46418c9e93bf61b348f8/users/dfa425e42202421c9e1c8564456c67f7/roles/d4088d1845a24fbd9bd8975ce0b1d3d0 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:11:17.243976 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcdb635-289f-43ad-b10f-20bd57720622 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:11:17.244542 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcdb635-289f-43ad-b10f-20bd57720622 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:11:17.245051 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcdb635-289f-43ad-b10f-20bd57720622 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:11:17.281072 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-dbcdb635-289f-43ad-b10f-20bd57720622 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:11:17.282687 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-dbcdb635-289f-43ad-b10f-20bd57720622 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:11:17.406853 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1108/2198] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:11:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:11:17.420159 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f469a8-1a32-46a7-8951-8310d70e6c06 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:11:17.423550 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-83f469a8-1a32-46a7-8951-8310d70e6c06 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:11:17.475708 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:11:17.540768 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f469a8-1a32-46a7-8951-8310d70e6c06 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:11:17.546945 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b7ab85-3719-44f9-a895-d21e2798e16c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0156a2f6f3794f479e76b85bdbbe5c26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:11:17.547624 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b7ab85-3719-44f9-a895-d21e2798e16c service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:11:17.548061 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b7ab85-3719-44f9-a895-d21e2798e16c service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:11:17.548346 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b7ab85-3719-44f9-a895-d21e2798e16c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:11:17.560438 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b7ab85-3719-44f9-a895-d21e2798e16c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:11:17.561980 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b7ab85-3719-44f9-a895-d21e2798e16c service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:11:17.575669 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1091/2199] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:11:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 15:11:18.482174 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc6a9be-cab2-4f04-92e9-4884874a4c55 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:11:18.483313 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc6a9be-cab2-4f04-92e9-4884874a4c55 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:11:18.483964 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc6a9be-cab2-4f04-92e9-4884874a4c55 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:11:18.485630 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1109/2200] 10.222.0.46 () {54 vars in 921 bytes} [Thu May 26 15:11:18 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 15:11:18.492246 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be45a9d-7867-4d62-ba68-74d50ba51559 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:11:18.493381 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be45a9d-7867-4d62-ba68-74d50ba51559 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:11:18.493381 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be45a9d-7867-4d62-ba68-74d50ba51559 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:11:18.602821 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-0be45a9d-7867-4d62-ba68-74d50ba51559 None None] MFA Rules not processed for user `d6639353c70449d1b3d10cd6487c3b00`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:11:18.607522 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-0be45a9d-7867-4d62-ba68-74d50ba51559 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:11:18.810254 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1092/2201] 10.222.0.46 () {58 vars in 1018 bytes} [Thu May 26 15:11:18 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 319 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:11:18.825285 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc6a9be-cab2-4f04-92e9-4884874a4c55 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:11:18.827197 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-1bc6a9be-cab2-4f04-92e9-4884874a4c55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:11:18.908807 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec9d277-f464-4f0e-9fc6-16429dfb580a None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:11:18.909000 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec9d277-f464-4f0e-9fc6-16429dfb580a None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:11:18.909879 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec9d277-f464-4f0e-9fc6-16429dfb580a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:11:18.934366 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:11:18.966387 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-9ec9d277-f464-4f0e-9fc6-16429dfb580a None None] MFA Rules not processed for user `dfa425e42202421c9e1c8564456c67f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:11:18.969650 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-9ec9d277-f464-4f0e-9fc6-16429dfb580a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:11:18.987898 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc6a9be-cab2-4f04-92e9-4884874a4c55 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:11:18.997806 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049af05e-b0b2-4c55-a69e-4b121443a8a7 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a436dbdee624f44ad76cca8416b0ad1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:11:18.998562 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049af05e-b0b2-4c55-a69e-4b121443a8a7 service glance] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:11:18.998771 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049af05e-b0b2-4c55-a69e-4b121443a8a7 service glance] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:11:18.999047 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049af05e-b0b2-4c55-a69e-4b121443a8a7 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:11:19.022632 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049af05e-b0b2-4c55-a69e-4b121443a8a7 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:11:19.024344 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049af05e-b0b2-4c55-a69e-4b121443a8a7 service glance] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:11:19.050412 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1110/2202] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:11:18 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 15:11:19.096570 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1093/2203] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:11:18 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:11:19.106663 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049af05e-b0b2-4c55-a69e-4b121443a8a7 service glance] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:11:19.108265 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-049af05e-b0b2-4c55-a69e-4b121443a8a7 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:11:19.213499 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a49d7a-e61b-4221-8ef9-a1229fe36f58 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:11:19.213824 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a49d7a-e61b-4221-8ef9-a1229fe36f58 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:11:19.214104 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a49d7a-e61b-4221-8ef9-a1229fe36f58 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:11:19.215659 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1094/2204] 10.222.0.46 () {54 vars in 921 bytes} [Thu May 26 15:11:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 15:11:19.223022 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a3c627-5856-4782-8f2f-d65d6a9c2fc4 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:11:19.223311 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a3c627-5856-4782-8f2f-d65d6a9c2fc4 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:11:19.223591 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a3c627-5856-4782-8f2f-d65d6a9c2fc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:11:19.258521 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-97a3c627-5856-4782-8f2f-d65d6a9c2fc4 None None] MFA Rules not processed for user `0156a2f6f3794f479e76b85bdbbe5c26`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:11:19.259746 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-97a3c627-5856-4782-8f2f-d65d6a9c2fc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:11:19.277993 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1095/2205] 10.222.0.46 () {58 vars in 1018 bytes} [Thu May 26 15:11:19 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:11:19.285930 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a3c627-5856-4782-8f2f-d65d6a9c2fc4 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:11:19.288133 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-97a3c627-5856-4782-8f2f-d65d6a9c2fc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:11:19.290611 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049af05e-b0b2-4c55-a69e-4b121443a8a7 service glance] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:11:19.296982 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06171fb3-5564-4297-bb68-487e1d47980f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5b5b35614a547d783e6e77ddb8c1c03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:11:19.297685 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06171fb3-5564-4297-bb68-487e1d47980f service cinder] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:11:19.297966 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06171fb3-5564-4297-bb68-487e1d47980f service cinder] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:11:19.298245 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06171fb3-5564-4297-bb68-487e1d47980f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:11:19.308815 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06171fb3-5564-4297-bb68-487e1d47980f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:11:19.309642 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:11:19.311026 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06171fb3-5564-4297-bb68-487e1d47980f service cinder] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:11:19.317196 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a3c627-5856-4782-8f2f-d65d6a9c2fc4 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:11:19.329696 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81425dd9-ef8c-4c4b-ac30-917d9bef0199 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0156a2f6f3794f479e76b85bdbbe5c26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:11:19.330382 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81425dd9-ef8c-4c4b-ac30-917d9bef0199 service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:11:19.330694 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81425dd9-ef8c-4c4b-ac30-917d9bef0199 service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:11:19.331753 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81425dd9-ef8c-4c4b-ac30-917d9bef0199 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:11:19.345682 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1111/2206] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:11:19 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 239 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 15:11:19.352146 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81425dd9-ef8c-4c4b-ac30-917d9bef0199 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:11:19.356823 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81425dd9-ef8c-4c4b-ac30-917d9bef0199 service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:11:19.388129 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1096/2207] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:11:19 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 15:11:19.554025 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e36ff8-7666-4497-900a-d2dfdb2727db None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:11:19.554593 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e36ff8-7666-4497-900a-d2dfdb2727db None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:11:19.554676 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e36ff8-7666-4497-900a-d2dfdb2727db None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:11:19.556454 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1112/2208] 10.222.0.46 () {54 vars in 921 bytes} [Thu May 26 15:11:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 26 15:11:19.563693 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cb0288-0c98-436e-841b-63acd80aab5a None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:11:19.564320 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cb0288-0c98-436e-841b-63acd80aab5a None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:11:19.564320 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cb0288-0c98-436e-841b-63acd80aab5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:11:19.624212 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-18cb0288-0c98-436e-841b-63acd80aab5a None None] MFA Rules not processed for user `d6639353c70449d1b3d10cd6487c3b00`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:11:19.625535 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-18cb0288-0c98-436e-841b-63acd80aab5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:11:19.660821 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1097/2209] 10.222.0.46 () {58 vars in 1018 bytes} [Thu May 26 15:11:19 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:11:19.668763 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e36ff8-7666-4497-900a-d2dfdb2727db None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:11:19.689683 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e36ff8-7666-4497-900a-d2dfdb2727db None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:11:19.699031 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cace6ed6-438e-4e71-9ae1-65170f9f258e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0156a2f6f3794f479e76b85bdbbe5c26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:11:19.700382 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cace6ed6-438e-4e71-9ae1-65170f9f258e service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:11:19.701059 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cace6ed6-438e-4e71-9ae1-65170f9f258e service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:11:19.701557 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cace6ed6-438e-4e71-9ae1-65170f9f258e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:11:19.730777 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cace6ed6-438e-4e71-9ae1-65170f9f258e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:11:19.732305 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cace6ed6-438e-4e71-9ae1-65170f9f258e service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:11:19.766210 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1113/2210] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:11:19 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 15:11:21.027382 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a9c24d-3021-4c68-99e2-976cb697bd9d None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:11:21.027917 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a9c24d-3021-4c68-99e2-976cb697bd9d None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:11:21.028038 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a9c24d-3021-4c68-99e2-976cb697bd9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:11:21.052086 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-87a9c24d-3021-4c68-99e2-976cb697bd9d None None] MFA Rules not processed for user `dfa425e42202421c9e1c8564456c67f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:11:21.053672 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-87a9c24d-3021-4c68-99e2-976cb697bd9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:11:21.077699 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1098/2211] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:11:21 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:11:25.005700 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e4cb4e-ee3c-4126-99b3-b5f27e9b47bc None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:11:25.006675 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e4cb4e-ee3c-4126-99b3-b5f27e9b47bc None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:11:25.007129 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e4cb4e-ee3c-4126-99b3-b5f27e9b47bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:11:25.048183 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-49e4cb4e-ee3c-4126-99b3-b5f27e9b47bc None None] MFA Rules not processed for user `8028c520d93449a6a087c281280ee13d`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:11:25.050044 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-49e4cb4e-ee3c-4126-99b3-b5f27e9b47bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:11:25.117630 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1114/2212] 10.222.0.46 () {58 vars in 1009 bytes} [Thu May 26 15:11:25 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:11:25.131762 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a9c24d-3021-4c68-99e2-976cb697bd9d None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:11:25.134505 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-87a9c24d-3021-4c68-99e2-976cb697bd9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:11:25.315247 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a9c24d-3021-4c68-99e2-976cb697bd9d None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:11:25.324792 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48de1543-ba6f-4559-9f93-2154e78aafd9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4bea272624b45d8ac2ad27bd07df885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:11:25.325490 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48de1543-ba6f-4559-9f93-2154e78aafd9 service swift] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:11:25.325797 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48de1543-ba6f-4559-9f93-2154e78aafd9 service swift] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:11:25.326209 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48de1543-ba6f-4559-9f93-2154e78aafd9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:11:25.337704 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48de1543-ba6f-4559-9f93-2154e78aafd9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:11:25.339164 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48de1543-ba6f-4559-9f93-2154e78aafd9 service swift] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:11:25.356076 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1099/2213] 10.222.0.46 () {60 vars in 1384 bytes} [Thu May 26 15:11:25 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 26 15:11:28.066962 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ca2899-e777-46ad-b021-fbb8a3c36c69 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:11:28.068085 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ca2899-e777-46ad-b021-fbb8a3c36c69 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:11:28.068680 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ca2899-e777-46ad-b021-fbb8a3c36c69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:11:28.139089 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-22ca2899-e777-46ad-b021-fbb8a3c36c69 None None] MFA Rules not processed for user `8028c520d93449a6a087c281280ee13d`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:11:28.140483 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-22ca2899-e777-46ad-b021-fbb8a3c36c69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:11:28.160270 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1115/2214] 10.222.0.46 () {58 vars in 967 bytes} [Thu May 26 15:11:28 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:11:28.178794 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48de1543-ba6f-4559-9f93-2154e78aafd9 service swift] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:11:28.221339 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48de1543-ba6f-4559-9f93-2154e78aafd9 service swift] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:11:28.228358 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beee6af9-8c65-4823-9e28-cc249c8fb5e4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4bea272624b45d8ac2ad27bd07df885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:11:28.229391 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beee6af9-8c65-4823-9e28-cc249c8fb5e4 service swift] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:11:28.230036 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beee6af9-8c65-4823-9e28-cc249c8fb5e4 service swift] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:11:28.230540 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beee6af9-8c65-4823-9e28-cc249c8fb5e4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:11:28.243461 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beee6af9-8c65-4823-9e28-cc249c8fb5e4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:11:28.245380 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beee6af9-8c65-4823-9e28-cc249c8fb5e4 service swift] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:11:28.257655 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1100/2215] 10.222.0.46 () {60 vars in 1384 bytes} [Thu May 26 15:11:28 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 79 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 26 15:11:30.523960 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ca2899-e777-46ad-b021-fbb8a3c36c69 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:11:30.525689 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-22ca2899-e777-46ad-b021-fbb8a3c36c69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:11:30.555790 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ca2899-e777-46ad-b021-fbb8a3c36c69 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:11:30.565829 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a718a139-0b36-4428-af9e-6ecd4062b272 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:11:30.567413 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a718a139-0b36-4428-af9e-6ecd4062b272 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:11:30.567699 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a718a139-0b36-4428-af9e-6ecd4062b272 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:11:30.567977 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a718a139-0b36-4428-af9e-6ecd4062b272 admin admin] PATH_INFO: `/v3/users/1c54e915f4014d58ba28d6e6d5dd6ab6` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:11:30.572212 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a718a139-0b36-4428-af9e-6ecd4062b272 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1c54e915f4014d58ba28d6e6d5dd6ab6)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:11:30.576675 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a718a139-0b36-4428-af9e-6ecd4062b272 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:11:30.589919 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:11:30.691534 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-a718a139-0b36-4428-af9e-6ecd4062b272 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1c54e915f4014d58ba28d6e6d5dd6ab6'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:11:30.709319 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-a718a139-0b36-4428-af9e-6ecd4062b272 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1c54e915f4014d58ba28d6e6d5dd6ab6'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:11:30.713913 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-a718a139-0b36-4428-af9e-6ecd4062b272 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1c54e915f4014d58ba28d6e6d5dd6ab6'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:11:30.757290 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-a718a139-0b36-4428-af9e-6ecd4062b272 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1c54e915f4014d58ba28d6e6d5dd6ab6'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:11:30.767301 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-a718a139-0b36-4428-af9e-6ecd4062b272 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1c54e915f4014d58ba28d6e6d5dd6ab6'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:11:30.777556 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1116/2216] 10.222.0.46 () {58 vars in 1223 bytes} [Thu May 26 15:11:30 2022] DELETE /identity/v3/users/1c54e915f4014d58ba28d6e6d5dd6ab6 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:11:31.337934 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beee6af9-8c65-4823-9e28-cc249c8fb5e4 service swift] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:11:31.352397 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beee6af9-8c65-4823-9e28-cc249c8fb5e4 service swift] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:11:31.420771 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8448854a-4f73-4df4-a1a1-d12200a3ed37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:11:31.421841 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8448854a-4f73-4df4-a1a1-d12200a3ed37 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:11:31.422201 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8448854a-4f73-4df4-a1a1-d12200a3ed37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:11:31.422569 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8448854a-4f73-4df4-a1a1-d12200a3ed37 admin admin] PATH_INFO: `/v3/projects/664448e66c54440f909ae6727ccc3deb` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:11:31.423984 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8448854a-4f73-4df4-a1a1-d12200a3ed37 admin admin] RBAC: Authorizing `identity:delete_project(project_id=664448e66c54440f909ae6727ccc3deb)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:11:31.426041 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8448854a-4f73-4df4-a1a1-d12200a3ed37 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:11:31.482062 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:11:31.484631 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-8448854a-4f73-4df4-a1a1-d12200a3ed37 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '664448e66c54440f909ae6727ccc3deb', 'request_id': 'req-8448854a-4f73-4df4-a1a1-d12200a3ed37'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:11:31.492556 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-8448854a-4f73-4df4-a1a1-d12200a3ed37 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '664448e66c54440f909ae6727ccc3deb', 'request_id': 'req-8448854a-4f73-4df4-a1a1-d12200a3ed37'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:11:31.499568 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-8448854a-4f73-4df4-a1a1-d12200a3ed37 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '664448e66c54440f909ae6727ccc3deb', 'request_id': 'req-8448854a-4f73-4df4-a1a1-d12200a3ed37'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:11:31.513441 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-8448854a-4f73-4df4-a1a1-d12200a3ed37 admin admin] The token cache is being invalidate because project 664448e66c54440f909ae6727ccc3deb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 15:11:31.513802 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-8448854a-4f73-4df4-a1a1-d12200a3ed37 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:11:31.514521 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-8448854a-4f73-4df4-a1a1-d12200a3ed37 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:11:31.516117 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1101/2217] 10.222.0.46 () {58 vars in 1232 bytes} [Thu May 26 15:11:31 2022] DELETE /identity/v3/projects/664448e66c54440f909ae6727ccc3deb => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:11:31.523573 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64c999e-187a-48f5-bca6-c17bfbd8edb6 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:11:31.523961 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64c999e-187a-48f5-bca6-c17bfbd8edb6 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:11:31.524124 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64c999e-187a-48f5-bca6-c17bfbd8edb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:11:31.543516 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-e64c999e-187a-48f5-bca6-c17bfbd8edb6 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:11:31.545047 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-e64c999e-187a-48f5-bca6-c17bfbd8edb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:11:31.615020 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1117/2218] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:11:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:11:31.624134 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bb76fb-6536-4205-a26c-1a9300e85297 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:11:31.624422 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bb76fb-6536-4205-a26c-1a9300e85297 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:11:31.624964 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bb76fb-6536-4205-a26c-1a9300e85297 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:11:31.646966 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-65bb76fb-6536-4205-a26c-1a9300e85297 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:11:31.648480 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-65bb76fb-6536-4205-a26c-1a9300e85297 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:11:31.658242 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1102/2219] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:11:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:11:31.662640 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64c999e-187a-48f5-bca6-c17bfbd8edb6 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:11:31.672669 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64c999e-187a-48f5-bca6-c17bfbd8edb6 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:11:31.677378 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25be383-6501-4f73-9bb8-253342bbe149 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:11:31.678083 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25be383-6501-4f73-9bb8-253342bbe149 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:11:31.678368 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25be383-6501-4f73-9bb8-253342bbe149 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:11:31.678647 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25be383-6501-4f73-9bb8-253342bbe149 admin admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:11:31.680003 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25be383-6501-4f73-9bb8-253342bbe149 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 15:11:31.680315 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25be383-6501-4f73-9bb8-253342bbe149 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:11:31.681955 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25be383-6501-4f73-9bb8-253342bbe149 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:11:31.699469 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1118/2220] 10.222.0.46 () {58 vars in 1152 bytes} [Thu May 26 15:11:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 15:11:31.700360 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bb76fb-6536-4205-a26c-1a9300e85297 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:11:31.710294 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bb76fb-6536-4205-a26c-1a9300e85297 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:11:31.715983 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69b9101-953b-42c8-a405-e8ef2abae3b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:11:31.716739 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69b9101-953b-42c8-a405-e8ef2abae3b7 admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:11:31.717134 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69b9101-953b-42c8-a405-e8ef2abae3b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:11:31.717778 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69b9101-953b-42c8-a405-e8ef2abae3b7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:11:31.718589 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69b9101-953b-42c8-a405-e8ef2abae3b7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:11:31.720611 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69b9101-953b-42c8-a405-e8ef2abae3b7 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:11:31.755572 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1103/2221] 10.222.0.46 () {60 vars in 1152 bytes} [Thu May 26 15:11:31 2022] POST /identity/v3/projects => generated 399 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 15:11:31.760053 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25be383-6501-4f73-9bb8-253342bbe149 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:11:31.783944 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25be383-6501-4f73-9bb8-253342bbe149 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:11:31.926795 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ebcbae-01dd-4017-b152-3647550c95fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:11:31.928464 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ebcbae-01dd-4017-b152-3647550c95fa admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:11:31.929369 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ebcbae-01dd-4017-b152-3647550c95fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:11:31.929770 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ebcbae-01dd-4017-b152-3647550c95fa admin admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:11:31.930561 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ebcbae-01dd-4017-b152-3647550c95fa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:11:31.939099 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ebcbae-01dd-4017-b152-3647550c95fa admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:11:31.977201 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1119/2222] 10.222.0.46 () {60 vars in 1143 bytes} [Thu May 26 15:11:31 2022] POST /identity/v3/users => generated 300 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 15:11:31.978057 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69b9101-953b-42c8-a405-e8ef2abae3b7 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:11:32.001330 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69b9101-953b-42c8-a405-e8ef2abae3b7 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:11:32.017191 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cb87f2-ce5f-41ba-8b36-7f074185dc4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:11:32.018011 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cb87f2-ce5f-41ba-8b36-7f074185dc4a admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:11:32.018307 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cb87f2-ce5f-41ba-8b36-7f074185dc4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:11:32.018587 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cb87f2-ce5f-41ba-8b36-7f074185dc4a admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:11:32.019329 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62cb87f2-ce5f-41ba-8b36-7f074185dc4a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:11:32.020962 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62cb87f2-ce5f-41ba-8b36-7f074185dc4a admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:11:32.033489 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1104/2223] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 15:11:31 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 15:11:32.036823 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ebcbae-01dd-4017-b152-3647550c95fa admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:11:32.043056 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:11:32.048926 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ebcbae-01dd-4017-b152-3647550c95fa admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:11:32.057720 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8ab1ad-9759-4873-81a2-196458b597d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:11:32.070340 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8ab1ad-9759-4873-81a2-196458b597d6 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:11:32.070974 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8ab1ad-9759-4873-81a2-196458b597d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:11:32.071377 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8ab1ad-9759-4873-81a2-196458b597d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:11:32.072328 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8ab1ad-9759-4873-81a2-196458b597d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:11:32.074338 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8ab1ad-9759-4873-81a2-196458b597d6 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:11:32.093848 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1120/2224] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 15:11:32 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 15:11:32.098110 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cb87f2-ce5f-41ba-8b36-7f074185dc4a admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:11:32.109309 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cb87f2-ce5f-41ba-8b36-7f074185dc4a admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:11:32.114079 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd8b8f0-f1ba-4bde-9bde-2348e4804352 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:11:32.114983 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd8b8f0-f1ba-4bde-9bde-2348e4804352 admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:11:32.115269 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd8b8f0-f1ba-4bde-9bde-2348e4804352 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:11:32.115547 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd8b8f0-f1ba-4bde-9bde-2348e4804352 admin admin] PATH_INFO: `/v3/projects/d6acdcfe35854c3b99c45315f0d5655d/users/65490371fb4b457d8c75fa9227a6002a/roles/d4088d1845a24fbd9bd8975ce0b1d3d0` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:11:32.137762 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd8b8f0-f1ba-4bde-9bde-2348e4804352 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d6acdcfe35854c3b99c45315f0d5655d, user_id=65490371fb4b457d8c75fa9227a6002a, role_id=d4088d1845a24fbd9bd8975ce0b1d3d0)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:11:32.140508 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd8b8f0-f1ba-4bde-9bde-2348e4804352 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:11:32.172918 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1105/2225] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 15:11:32 2022] PUT /identity/v3/projects/d6acdcfe35854c3b99c45315f0d5655d/users/65490371fb4b457d8c75fa9227a6002a/roles/d4088d1845a24fbd9bd8975ce0b1d3d0 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:11:32.183393 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9104ea-0e76-490a-a4fd-deb1bc37cf71 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:11:32.183843 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9104ea-0e76-490a-a4fd-deb1bc37cf71 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:11:32.184139 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a9104ea-0e76-490a-a4fd-deb1bc37cf71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:11:32.215897 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-4a9104ea-0e76-490a-a4fd-deb1bc37cf71 None None] MFA Rules not processed for user `65490371fb4b457d8c75fa9227a6002a`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:11:32.217130 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-4a9104ea-0e76-490a-a4fd-deb1bc37cf71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:11:32.333887 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1121/2226] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:11:32 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:11:32.342251 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd8b8f0-f1ba-4bde-9bde-2348e4804352 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:11:32.345086 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-1fd8b8f0-f1ba-4bde-9bde-2348e4804352 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:11:32.457824 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd8b8f0-f1ba-4bde-9bde-2348e4804352 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:11:32.462148 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcdf520-a0ef-41c0-87d0-40e35c9064f8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5b5b35614a547d783e6e77ddb8c1c03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:11:32.463034 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcdf520-a0ef-41c0-87d0-40e35c9064f8 service cinder] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:11:32.463746 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcdf520-a0ef-41c0-87d0-40e35c9064f8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:11:32.480463 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcdf520-a0ef-41c0-87d0-40e35c9064f8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:11:32.486052 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dcdf520-a0ef-41c0-87d0-40e35c9064f8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:11:32.488620 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dcdf520-a0ef-41c0-87d0-40e35c9064f8 service cinder] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:11:32.538592 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1106/2227] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:11:32 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 15:11:39.732031 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9104ea-0e76-490a-a4fd-deb1bc37cf71 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:11:39.801078 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a9104ea-0e76-490a-a4fd-deb1bc37cf71 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:11:39.806192 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1b9f83-9e34-4ac3-9b15-59e09fe2dc3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:11:39.807055 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1b9f83-9e34-4ac3-9b15-59e09fe2dc3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:11:39.807342 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1b9f83-9e34-4ac3-9b15-59e09fe2dc3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:11:39.807621 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1b9f83-9e34-4ac3-9b15-59e09fe2dc3d admin admin] PATH_INFO: `/v3/users/65490371fb4b457d8c75fa9227a6002a` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:11:39.808806 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1b9f83-9e34-4ac3-9b15-59e09fe2dc3d admin admin] RBAC: Authorizing `identity:delete_user(user_id=65490371fb4b457d8c75fa9227a6002a)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:11:39.810781 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1b9f83-9e34-4ac3-9b15-59e09fe2dc3d admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:11:39.859818 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-da1b9f83-9e34-4ac3-9b15-59e09fe2dc3d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '65490371fb4b457d8c75fa9227a6002a'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:11:39.869043 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-da1b9f83-9e34-4ac3-9b15-59e09fe2dc3d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '65490371fb4b457d8c75fa9227a6002a'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:11:39.869699 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-da1b9f83-9e34-4ac3-9b15-59e09fe2dc3d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '65490371fb4b457d8c75fa9227a6002a'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:11:39.883600 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-da1b9f83-9e34-4ac3-9b15-59e09fe2dc3d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '65490371fb4b457d8c75fa9227a6002a'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:11:39.891767 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-da1b9f83-9e34-4ac3-9b15-59e09fe2dc3d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '65490371fb4b457d8c75fa9227a6002a'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:11:39.894328 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:11:39.905755 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7b101a-b187-4f3d-b616-3c5f0c27aed8 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:11:39.906347 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7b101a-b187-4f3d-b616-3c5f0c27aed8 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:11:39.906347 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7b101a-b187-4f3d-b616-3c5f0c27aed8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:11:39.909886 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1122/2228] 10.222.0.46 () {58 vars in 1223 bytes} [Thu May 26 15:11:39 2022] DELETE /identity/v3/users/65490371fb4b457d8c75fa9227a6002a => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:11:39.936187 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-7b7b101a-b187-4f3d-b616-3c5f0c27aed8 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:11:39.937526 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-7b7b101a-b187-4f3d-b616-3c5f0c27aed8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:11:40.024283 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1107/2229] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:11:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 15:11:40.060741 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1b9f83-9e34-4ac3-9b15-59e09fe2dc3d admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:11:40.062978 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-da1b9f83-9e34-4ac3-9b15-59e09fe2dc3d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:11:40.138382 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1b9f83-9e34-4ac3-9b15-59e09fe2dc3d admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:11:40.144557 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60a1587-1884-41d9-a35c-c2d98def42b4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0156a2f6f3794f479e76b85bdbbe5c26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:11:40.145226 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60a1587-1884-41d9-a35c-c2d98def42b4 service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:11:40.145537 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60a1587-1884-41d9-a35c-c2d98def42b4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:11:40.145817 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60a1587-1884-41d9-a35c-c2d98def42b4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:11:40.160207 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60a1587-1884-41d9-a35c-c2d98def42b4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:11:40.161894 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60a1587-1884-41d9-a35c-c2d98def42b4 service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:11:40.173163 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1123/2230] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:11:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 15:11:40.970139 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7b101a-b187-4f3d-b616-3c5f0c27aed8 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:11:40.972769 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-7b7b101a-b187-4f3d-b616-3c5f0c27aed8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:11:40.988072 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7b101a-b187-4f3d-b616-3c5f0c27aed8 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:11:40.993522 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae6258f-e284-41e5-8801-f3fcafad7db8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:11:40.994554 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae6258f-e284-41e5-8801-f3fcafad7db8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:11:40.995013 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae6258f-e284-41e5-8801-f3fcafad7db8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:11:40.995361 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae6258f-e284-41e5-8801-f3fcafad7db8 admin admin] PATH_INFO: `/v3/projects/d6acdcfe35854c3b99c45315f0d5655d` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:11:40.997612 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae6258f-e284-41e5-8801-f3fcafad7db8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d6acdcfe35854c3b99c45315f0d5655d)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:11:40.999731 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae6258f-e284-41e5-8801-f3fcafad7db8 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:11:41.016001 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=94,keystone:DELETE=3,keystone:INSERT=3 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:11:41.073140 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-aae6258f-e284-41e5-8801-f3fcafad7db8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd6acdcfe35854c3b99c45315f0d5655d', 'request_id': 'req-aae6258f-e284-41e5-8801-f3fcafad7db8'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:11:41.079884 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-aae6258f-e284-41e5-8801-f3fcafad7db8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd6acdcfe35854c3b99c45315f0d5655d', 'request_id': 'req-aae6258f-e284-41e5-8801-f3fcafad7db8'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:11:41.087294 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-aae6258f-e284-41e5-8801-f3fcafad7db8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd6acdcfe35854c3b99c45315f0d5655d', 'request_id': 'req-aae6258f-e284-41e5-8801-f3fcafad7db8'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:11:41.105090 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-aae6258f-e284-41e5-8801-f3fcafad7db8 admin admin] The token cache is being invalidate because project d6acdcfe35854c3b99c45315f0d5655d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 15:11:41.105481 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-aae6258f-e284-41e5-8801-f3fcafad7db8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:11:41.106231 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-aae6258f-e284-41e5-8801-f3fcafad7db8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:11:41.108038 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1108/2231] 10.222.0.46 () {58 vars in 1232 bytes} [Thu May 26 15:11:40 2022] DELETE /identity/v3/projects/d6acdcfe35854c3b99c45315f0d5655d => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:11:41.113026 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca08799-9b69-43fd-b83a-57ea98a0283b None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:11:41.113914 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca08799-9b69-43fd-b83a-57ea98a0283b None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:11:41.114188 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca08799-9b69-43fd-b83a-57ea98a0283b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:11:41.133157 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-5ca08799-9b69-43fd-b83a-57ea98a0283b None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:11:41.134400 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-5ca08799-9b69-43fd-b83a-57ea98a0283b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:11:41.200557 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1124/2232] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:11:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:11:41.209360 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f48d6f-2ade-4af0-9175-c7e39628a95b None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:11:41.209783 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f48d6f-2ade-4af0-9175-c7e39628a95b None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:11:41.210136 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f48d6f-2ade-4af0-9175-c7e39628a95b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:11:41.230234 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-e9f48d6f-2ade-4af0-9175-c7e39628a95b None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:11:41.231619 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-e9f48d6f-2ade-4af0-9175-c7e39628a95b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:11:41.241994 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1109/2233] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:11:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 15:11:41.247153 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca08799-9b69-43fd-b83a-57ea98a0283b None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:11:41.261540 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca08799-9b69-43fd-b83a-57ea98a0283b None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:11:41.266312 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead37283-ff1d-4726-983d-7a1441c2be60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:11:41.267146 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead37283-ff1d-4726-983d-7a1441c2be60 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:11:41.267435 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead37283-ff1d-4726-983d-7a1441c2be60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:11:41.267704 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead37283-ff1d-4726-983d-7a1441c2be60 admin admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:11:41.268753 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead37283-ff1d-4726-983d-7a1441c2be60 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 15:11:41.269210 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead37283-ff1d-4726-983d-7a1441c2be60 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:11:41.271012 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead37283-ff1d-4726-983d-7a1441c2be60 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:11:41.290323 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1125/2234] 10.222.0.46 () {58 vars in 1152 bytes} [Thu May 26 15:11:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 15:11:41.292098 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f48d6f-2ade-4af0-9175-c7e39628a95b None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:11:41.307201 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f48d6f-2ade-4af0-9175-c7e39628a95b None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:11:41.314459 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b25009-3bdb-4dee-b937-e81cb84dd997 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:11:41.315288 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b25009-3bdb-4dee-b937-e81cb84dd997 admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:11:41.315719 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b25009-3bdb-4dee-b937-e81cb84dd997 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:11:41.316046 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b25009-3bdb-4dee-b937-e81cb84dd997 admin admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:11:41.317219 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b25009-3bdb-4dee-b937-e81cb84dd997 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:11:41.320409 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b25009-3bdb-4dee-b937-e81cb84dd997 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:11:41.352351 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1110/2235] 10.222.0.46 () {60 vars in 1152 bytes} [Thu May 26 15:11:41 2022] POST /identity/v3/projects => generated 373 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 15:11:41.356673 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead37283-ff1d-4726-983d-7a1441c2be60 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:11:41.371189 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead37283-ff1d-4726-983d-7a1441c2be60 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:11:41.445186 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f698909-15af-4212-aae3-414321c6dafe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:11:41.446299 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f698909-15af-4212-aae3-414321c6dafe admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:11:41.446739 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f698909-15af-4212-aae3-414321c6dafe admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:11:41.447171 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f698909-15af-4212-aae3-414321c6dafe admin admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:11:41.448102 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f698909-15af-4212-aae3-414321c6dafe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:11:41.450295 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f698909-15af-4212-aae3-414321c6dafe admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:11:41.480009 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1126/2236] 10.222.0.46 () {60 vars in 1143 bytes} [Thu May 26 15:11:41 2022] POST /identity/v3/users => generated 287 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 15:11:41.480892 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b25009-3bdb-4dee-b937-e81cb84dd997 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:11:41.494080 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b25009-3bdb-4dee-b937-e81cb84dd997 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:11:41.499216 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a507bf0-4c2f-4f21-b367-8717d4d84433 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:11:41.499891 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a507bf0-4c2f-4f21-b367-8717d4d84433 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:11:41.500177 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a507bf0-4c2f-4f21-b367-8717d4d84433 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:11:41.500460 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a507bf0-4c2f-4f21-b367-8717d4d84433 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:11:41.501193 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a507bf0-4c2f-4f21-b367-8717d4d84433 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:11:41.503002 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a507bf0-4c2f-4f21-b367-8717d4d84433 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:11:41.520353 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1111/2237] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 15:11:41 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 15:11:41.522000 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f698909-15af-4212-aae3-414321c6dafe admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:11:41.533778 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f698909-15af-4212-aae3-414321c6dafe admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:11:41.538602 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9346c4f-1855-4876-872a-286b2b871ad5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:11:41.539970 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9346c4f-1855-4876-872a-286b2b871ad5 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:11:41.540380 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9346c4f-1855-4876-872a-286b2b871ad5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:11:41.540750 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9346c4f-1855-4876-872a-286b2b871ad5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:11:41.541724 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9346c4f-1855-4876-872a-286b2b871ad5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:11:41.543366 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9346c4f-1855-4876-872a-286b2b871ad5 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:11:41.557098 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1127/2238] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 15:11:41 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 15:11:41.561270 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a507bf0-4c2f-4f21-b367-8717d4d84433 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:11:41.573399 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a507bf0-4c2f-4f21-b367-8717d4d84433 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:11:41.581353 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a705094b-a646-42bf-b041-66e54b2348bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:11:41.582315 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a705094b-a646-42bf-b041-66e54b2348bd admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:11:41.582599 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a705094b-a646-42bf-b041-66e54b2348bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:11:41.582879 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a705094b-a646-42bf-b041-66e54b2348bd admin admin] PATH_INFO: `/v3/projects/9c5cd1d25e6146bd93bd90286f668b9d/users/e4afc097467f4c26a6d4da0e48b8303f/roles/d4088d1845a24fbd9bd8975ce0b1d3d0` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:11:41.600647 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a705094b-a646-42bf-b041-66e54b2348bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c5cd1d25e6146bd93bd90286f668b9d, user_id=e4afc097467f4c26a6d4da0e48b8303f, role_id=d4088d1845a24fbd9bd8975ce0b1d3d0)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:11:41.610097 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a705094b-a646-42bf-b041-66e54b2348bd admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:11:41.634266 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1112/2239] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 15:11:41 2022] PUT /identity/v3/projects/9c5cd1d25e6146bd93bd90286f668b9d/users/e4afc097467f4c26a6d4da0e48b8303f/roles/d4088d1845a24fbd9bd8975ce0b1d3d0 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:11:41.644865 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adc4bd9-e4ba-4071-a682-6875757520ac None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:11:41.645154 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adc4bd9-e4ba-4071-a682-6875757520ac None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:11:41.645524 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adc4bd9-e4ba-4071-a682-6875757520ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:11:41.667892 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-3adc4bd9-e4ba-4071-a682-6875757520ac None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:11:41.669590 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-3adc4bd9-e4ba-4071-a682-6875757520ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:11:41.674063 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:11:41.736659 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1128/2240] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:11:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:11:41.746719 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a705094b-a646-42bf-b041-66e54b2348bd admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:11:41.748715 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-a705094b-a646-42bf-b041-66e54b2348bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:11:41.808749 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a705094b-a646-42bf-b041-66e54b2348bd admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:11:41.813079 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef73ab6b-2c2b-4bee-b34a-2d71a71d6fe5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0156a2f6f3794f479e76b85bdbbe5c26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:11:41.813897 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef73ab6b-2c2b-4bee-b34a-2d71a71d6fe5 service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:11:41.814281 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef73ab6b-2c2b-4bee-b34a-2d71a71d6fe5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:11:41.814683 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef73ab6b-2c2b-4bee-b34a-2d71a71d6fe5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:11:41.824375 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef73ab6b-2c2b-4bee-b34a-2d71a71d6fe5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:11:41.825860 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef73ab6b-2c2b-4bee-b34a-2d71a71d6fe5 service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:11:41.835391 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1113/2241] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:11:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 15:11:48.222562 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d93d305-df6b-48fa-a394-3f7324287435 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:11:48.223092 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d93d305-df6b-48fa-a394-3f7324287435 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:11:48.223092 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d93d305-df6b-48fa-a394-3f7324287435 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:11:48.253797 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-7d93d305-df6b-48fa-a394-3f7324287435 None None] MFA Rules not processed for user `e4afc097467f4c26a6d4da0e48b8303f`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:11:48.255239 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-7d93d305-df6b-48fa-a394-3f7324287435 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:11:48.331408 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1129/2242] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:11:48 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:11:48.336220 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef73ab6b-2c2b-4bee-b34a-2d71a71d6fe5 service neutron] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:11:48.349080 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef73ab6b-2c2b-4bee-b34a-2d71a71d6fe5 service neutron] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:11:48.355238 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473b6ebb-9584-4351-b0b8-29f90bb2f58b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:11:48.356003 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473b6ebb-9584-4351-b0b8-29f90bb2f58b admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:11:48.356297 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473b6ebb-9584-4351-b0b8-29f90bb2f58b admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:11:48.356613 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473b6ebb-9584-4351-b0b8-29f90bb2f58b admin admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:11:48.358255 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473b6ebb-9584-4351-b0b8-29f90bb2f58b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:11:48.360191 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473b6ebb-9584-4351-b0b8-29f90bb2f58b admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:11:48.387127 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1114/2243] 10.222.0.46 () {60 vars in 1152 bytes} [Thu May 26 15:11:48 2022] POST /identity/v3/projects => generated 373 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 15:11:48.391321 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d93d305-df6b-48fa-a394-3f7324287435 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:11:48.402881 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d93d305-df6b-48fa-a394-3f7324287435 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:11:48.467369 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc1df24-afb9-4581-9bcb-bd2dfa561dce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:11:48.468127 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc1df24-afb9-4581-9bcb-bd2dfa561dce admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:11:48.468338 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc1df24-afb9-4581-9bcb-bd2dfa561dce admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:11:48.468764 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc1df24-afb9-4581-9bcb-bd2dfa561dce admin admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:11:48.469532 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc1df24-afb9-4581-9bcb-bd2dfa561dce admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:11:48.471615 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc1df24-afb9-4581-9bcb-bd2dfa561dce admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:11:48.497387 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1130/2244] 10.222.0.46 () {60 vars in 1143 bytes} [Thu May 26 15:11:48 2022] POST /identity/v3/users => generated 293 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 15:11:48.501380 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473b6ebb-9584-4351-b0b8-29f90bb2f58b admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:11:48.512342 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473b6ebb-9584-4351-b0b8-29f90bb2f58b admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:11:48.517740 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3335ace-07ea-4347-a38c-41a8b7f3f00d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:11:48.518433 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3335ace-07ea-4347-a38c-41a8b7f3f00d admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:11:48.518719 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3335ace-07ea-4347-a38c-41a8b7f3f00d admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:11:48.519001 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3335ace-07ea-4347-a38c-41a8b7f3f00d admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:11:48.519741 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3335ace-07ea-4347-a38c-41a8b7f3f00d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:11:48.521413 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3335ace-07ea-4347-a38c-41a8b7f3f00d admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:11:48.536522 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1115/2245] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 15:11:48 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 15:11:48.537897 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc1df24-afb9-4581-9bcb-bd2dfa561dce admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:11:48.549208 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc1df24-afb9-4581-9bcb-bd2dfa561dce admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:11:48.554152 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414a4df0-9ec5-4fce-99b0-b8db533446ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:11:48.555087 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414a4df0-9ec5-4fce-99b0-b8db533446ca admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:11:48.555376 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414a4df0-9ec5-4fce-99b0-b8db533446ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:11:48.555656 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414a4df0-9ec5-4fce-99b0-b8db533446ca admin admin] PATH_INFO: `/v3/projects/6307f64212a64a84b5369682457e94db/users/ea4c4d48b2174f24a937002d5eb19d77/roles/a2e551773cbd4df084020676865b8222` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:11:48.574248 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414a4df0-9ec5-4fce-99b0-b8db533446ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=6307f64212a64a84b5369682457e94db, user_id=ea4c4d48b2174f24a937002d5eb19d77, role_id=a2e551773cbd4df084020676865b8222)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:11:48.577171 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414a4df0-9ec5-4fce-99b0-b8db533446ca admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:11:48.594280 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1131/2246] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 15:11:48 2022] PUT /identity/v3/projects/6307f64212a64a84b5369682457e94db/users/ea4c4d48b2174f24a937002d5eb19d77/roles/a2e551773cbd4df084020676865b8222 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:11:48.599728 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3335ace-07ea-4347-a38c-41a8b7f3f00d admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:11:48.607454 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:11:48.611566 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3335ace-07ea-4347-a38c-41a8b7f3f00d admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:11:48.676420 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f41054-23c8-4e30-b341-68b991c6d07f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:11:48.677189 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f41054-23c8-4e30-b341-68b991c6d07f admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:11:48.677582 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f41054-23c8-4e30-b341-68b991c6d07f admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:11:48.677927 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f41054-23c8-4e30-b341-68b991c6d07f admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:11:48.678717 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f41054-23c8-4e30-b341-68b991c6d07f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:11:48.680413 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f41054-23c8-4e30-b341-68b991c6d07f admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:11:48.695233 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1116/2247] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 15:11:48 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 15:11:48.698429 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414a4df0-9ec5-4fce-99b0-b8db533446ca admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:11:48.706390 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:11:48.713896 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414a4df0-9ec5-4fce-99b0-b8db533446ca admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:11:48.721965 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837e7b0b-71c5-48b1-8bdb-4df9d58f7878 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:11:48.722918 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837e7b0b-71c5-48b1-8bdb-4df9d58f7878 admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:11:48.723910 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837e7b0b-71c5-48b1-8bdb-4df9d58f7878 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:11:48.724164 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837e7b0b-71c5-48b1-8bdb-4df9d58f7878 admin admin] PATH_INFO: `/v3/projects/6307f64212a64a84b5369682457e94db/users/ea4c4d48b2174f24a937002d5eb19d77/roles/a2e551773cbd4df084020676865b8222` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:11:48.728144 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837e7b0b-71c5-48b1-8bdb-4df9d58f7878 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6307f64212a64a84b5369682457e94db, user_id=ea4c4d48b2174f24a937002d5eb19d77, role_id=a2e551773cbd4df084020676865b8222)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:11:48.732316 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837e7b0b-71c5-48b1-8bdb-4df9d58f7878 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:11:48.752179 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1132/2248] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 15:11:48 2022] PUT /identity/v3/projects/6307f64212a64a84b5369682457e94db/users/ea4c4d48b2174f24a937002d5eb19d77/roles/a2e551773cbd4df084020676865b8222 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:11:52.652392 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33867a77-6703-466e-9aba-91108bd48ade None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:11:52.653191 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33867a77-6703-466e-9aba-91108bd48ade None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:11:52.653191 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33867a77-6703-466e-9aba-91108bd48ade None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:11:52.691638 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-33867a77-6703-466e-9aba-91108bd48ade None None] MFA Rules not processed for user `8028c520d93449a6a087c281280ee13d`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:11:52.692958 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-33867a77-6703-466e-9aba-91108bd48ade None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:11:52.829893 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1117/2249] 10.222.0.46 () {58 vars in 967 bytes} [Thu May 26 15:11:52 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:11:52.846776 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837e7b0b-71c5-48b1-8bdb-4df9d58f7878 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:11:52.848736 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-837e7b0b-71c5-48b1-8bdb-4df9d58f7878 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:11:52.952805 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837e7b0b-71c5-48b1-8bdb-4df9d58f7878 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:11:52.961097 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a546116f-a5ab-4811-8fd6-a25867a171d6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4bea272624b45d8ac2ad27bd07df885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:11:52.962141 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a546116f-a5ab-4811-8fd6-a25867a171d6 service swift] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:11:52.962559 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a546116f-a5ab-4811-8fd6-a25867a171d6 service swift] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:11:52.962967 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a546116f-a5ab-4811-8fd6-a25867a171d6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:11:52.977190 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a546116f-a5ab-4811-8fd6-a25867a171d6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:11:52.978972 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a546116f-a5ab-4811-8fd6-a25867a171d6 service swift] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:11:52.989512 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1133/2250] 10.222.0.46 () {60 vars in 1384 bytes} [Thu May 26 15:11:52 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 26 15:11:55.267696 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c42cc50-58ad-46d3-bd2b-90a4483f9900 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:11:55.268255 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c42cc50-58ad-46d3-bd2b-90a4483f9900 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:11:55.268255 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c42cc50-58ad-46d3-bd2b-90a4483f9900 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:11:55.296959 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-8c42cc50-58ad-46d3-bd2b-90a4483f9900 None None] MFA Rules not processed for user `ea4c4d48b2174f24a937002d5eb19d77`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:11:55.298241 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-8c42cc50-58ad-46d3-bd2b-90a4483f9900 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:11:55.381647 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1118/2251] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:11:55 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:11:55.389333 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a546116f-a5ab-4811-8fd6-a25867a171d6 service swift] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:11:55.390864 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-a546116f-a5ab-4811-8fd6-a25867a171d6 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:11:55.493564 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a546116f-a5ab-4811-8fd6-a25867a171d6 service swift] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:11:55.501010 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d19f63-699b-43e5-a7d2-9781c173be05 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6639353c70449d1b3d10cd6487c3b00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:11:55.502037 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d19f63-699b-43e5-a7d2-9781c173be05 service nova] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:11:55.502628 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d19f63-699b-43e5-a7d2-9781c173be05 service nova] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:11:55.503098 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d19f63-699b-43e5-a7d2-9781c173be05 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:11:55.572519 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d19f63-699b-43e5-a7d2-9781c173be05 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:11:55.574193 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d19f63-699b-43e5-a7d2-9781c173be05 service nova] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:11:55.583952 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1134/2252] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:11:55 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 15:11:56.640296 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87d92a4-5bca-44e8-8f5c-ac58a508daf3 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:11:56.640804 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87d92a4-5bca-44e8-8f5c-ac58a508daf3 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:11:56.640886 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87d92a4-5bca-44e8-8f5c-ac58a508daf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:11:56.664316 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-f87d92a4-5bca-44e8-8f5c-ac58a508daf3 None None] MFA Rules not processed for user `e4afc097467f4c26a6d4da0e48b8303f`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:11:56.665592 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-f87d92a4-5bca-44e8-8f5c-ac58a508daf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:11:56.672663 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1119/2253] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:11:56 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:12:05.579330 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=96,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:12:06.656492 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=89 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:12:17.997354 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306e99c9-2b72-49af-b8c4-23b234129c2a None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:12:17.997886 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306e99c9-2b72-49af-b8c4-23b234129c2a None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:12:17.998150 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-306e99c9-2b72-49af-b8c4-23b234129c2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:12:18.015254 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-306e99c9-2b72-49af-b8c4-23b234129c2a None None] MFA Rules not processed for user `27172e53a12a437c9d57a75a95727d71`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:12:18.016444 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-306e99c9-2b72-49af-b8c4-23b234129c2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:12:18.072403 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1135/2254] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:12:17 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:12:19.130706 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87d92a4-5bca-44e8-8f5c-ac58a508daf3 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:12:19.132234 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-f87d92a4-5bca-44e8-8f5c-ac58a508daf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:12:19.222299 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87d92a4-5bca-44e8-8f5c-ac58a508daf3 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:12:19.227978 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30a53af-4ba0-4d5b-b527-d865943e91a2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf55a00e3b084aa3b22c25a90e6a773c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:12:19.228630 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30a53af-4ba0-4d5b-b527-d865943e91a2 service placement] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:12:19.228912 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30a53af-4ba0-4d5b-b527-d865943e91a2 service placement] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:12:19.229190 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30a53af-4ba0-4d5b-b527-d865943e91a2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:12:19.231578 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-e30a53af-4ba0-4d5b-b527-d865943e91a2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:12:19.243559 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30a53af-4ba0-4d5b-b527-d865943e91a2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:12:19.245189 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30a53af-4ba0-4d5b-b527-d865943e91a2 service placement] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:12:19.256161 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1120/2255] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:12:19 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 15:12:28.065801 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=25 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:12:29.251771 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=41 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:12:49.079653 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306e99c9-2b72-49af-b8c4-23b234129c2a None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:12:49.081064 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-306e99c9-2b72-49af-b8c4-23b234129c2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:12:49.137692 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-306e99c9-2b72-49af-b8c4-23b234129c2a None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:12:49.141801 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f84241-4c42-420d-a3a6-91a79547780f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0156a2f6f3794f479e76b85bdbbe5c26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:12:49.142442 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f84241-4c42-420d-a3a6-91a79547780f service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:12:49.142725 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f84241-4c42-420d-a3a6-91a79547780f service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:12:49.143003 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f84241-4c42-420d-a3a6-91a79547780f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:12:49.151974 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f84241-4c42-420d-a3a6-91a79547780f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:12:49.153393 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f84241-4c42-420d-a3a6-91a79547780f service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:12:49.163332 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1136/2256] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:12:49 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 15:12:58.812933 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b33711-a7d2-4d6e-9fe5-245ab1193532 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:12:58.813443 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b33711-a7d2-4d6e-9fe5-245ab1193532 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:12:58.813443 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b33711-a7d2-4d6e-9fe5-245ab1193532 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:12:58.831189 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-f8b33711-a7d2-4d6e-9fe5-245ab1193532 None None] MFA Rules not processed for user `dfa425e42202421c9e1c8564456c67f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:12:58.832392 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-f8b33711-a7d2-4d6e-9fe5-245ab1193532 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:12:58.891486 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1121/2257] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:12:58 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:12:59.159153 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=26 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:13:08.882840 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=25 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:13:11.215698 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be65ff6-0e63-494e-bd52-aaff201e6514 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:13:11.216228 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be65ff6-0e63-494e-bd52-aaff201e6514 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:13:11.216905 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be65ff6-0e63-494e-bd52-aaff201e6514 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:13:11.235231 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-6be65ff6-0e63-494e-bd52-aaff201e6514 None None] MFA Rules not processed for user `64aeada36ebe460e9969f1f048651e6d`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:13:11.236456 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-6be65ff6-0e63-494e-bd52-aaff201e6514 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:13:11.296608 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1137/2258] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:13:11 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:13:19.227880 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf160a0-cf72-4908-b9f3-5a5a4a1342da None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:13:19.228420 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf160a0-cf72-4908-b9f3-5a5a4a1342da None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:13:19.228420 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf160a0-cf72-4908-b9f3-5a5a4a1342da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:13:19.246654 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-fdf160a0-cf72-4908-b9f3-5a5a4a1342da None None] MFA Rules not processed for user `2ce5b5d5a12743bc80a4b191c24ff52f`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:13:19.247885 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-fdf160a0-cf72-4908-b9f3-5a5a4a1342da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:13:19.306336 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1122/2259] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:13:19 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:13:21.289364 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=25 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:13:29.299240 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=25 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:13:48.440487 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be65ff6-0e63-494e-bd52-aaff201e6514 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:13:48.450558 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be65ff6-0e63-494e-bd52-aaff201e6514 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:13:48.454826 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed65f3ff-1a25-4d06-a82c-354a1b820c52 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0156a2f6f3794f479e76b85bdbbe5c26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:13:48.455484 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed65f3ff-1a25-4d06-a82c-354a1b820c52 service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:13:48.455774 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed65f3ff-1a25-4d06-a82c-354a1b820c52 service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:13:48.456063 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed65f3ff-1a25-4d06-a82c-354a1b820c52 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:13:48.457928 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-ed65f3ff-1a25-4d06-a82c-354a1b820c52 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:13:48.472511 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed65f3ff-1a25-4d06-a82c-354a1b820c52 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:13:48.474035 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed65f3ff-1a25-4d06-a82c-354a1b820c52 service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:13:48.485807 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1138/2260] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:13:48 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 15:13:52.056931 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf160a0-cf72-4908-b9f3-5a5a4a1342da None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:13:52.066532 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf160a0-cf72-4908-b9f3-5a5a4a1342da None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:13:52.070308 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915da249-fdad-4b9b-8b51-fbda844e0eed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0156a2f6f3794f479e76b85bdbbe5c26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:13:52.070976 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915da249-fdad-4b9b-8b51-fbda844e0eed service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:13:52.071260 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915da249-fdad-4b9b-8b51-fbda844e0eed service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:13:52.071539 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915da249-fdad-4b9b-8b51-fbda844e0eed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:13:52.073303 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-915da249-fdad-4b9b-8b51-fbda844e0eed service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:13:52.087570 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915da249-fdad-4b9b-8b51-fbda844e0eed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:13:52.089012 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915da249-fdad-4b9b-8b51-fbda844e0eed service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:13:52.100996 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1123/2261] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:13:52 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 15:13:58.480588 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=6 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:14:02.095796 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=6 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:14:07.037837 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed65f3ff-1a25-4d06-a82c-354a1b820c52 service neutron] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:14:07.039404 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-ed65f3ff-1a25-4d06-a82c-354a1b820c52 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:14:07.111000 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed65f3ff-1a25-4d06-a82c-354a1b820c52 service neutron] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:14:07.115550 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb939ff-dc25-4201-b585-2941086a3cb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:14:07.116468 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb939ff-dc25-4201-b585-2941086a3cb4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:14:07.117090 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb939ff-dc25-4201-b585-2941086a3cb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:14:07.117386 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb939ff-dc25-4201-b585-2941086a3cb4 admin admin] PATH_INFO: `/v3/users/dfa425e42202421c9e1c8564456c67f7` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:14:07.118687 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb939ff-dc25-4201-b585-2941086a3cb4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dfa425e42202421c9e1c8564456c67f7)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:14:07.120898 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb939ff-dc25-4201-b585-2941086a3cb4 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:14:07.178398 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-dcb939ff-dc25-4201-b585-2941086a3cb4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dfa425e42202421c9e1c8564456c67f7'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:14:07.187988 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-dcb939ff-dc25-4201-b585-2941086a3cb4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dfa425e42202421c9e1c8564456c67f7'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:14:07.189039 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-dcb939ff-dc25-4201-b585-2941086a3cb4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dfa425e42202421c9e1c8564456c67f7'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:14:07.208791 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-dcb939ff-dc25-4201-b585-2941086a3cb4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dfa425e42202421c9e1c8564456c67f7'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:14:07.221776 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-dcb939ff-dc25-4201-b585-2941086a3cb4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dfa425e42202421c9e1c8564456c67f7'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:14:07.231445 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1139/2262] 10.222.0.46 () {58 vars in 1223 bytes} [Thu May 26 15:14:07 2022] DELETE /identity/v3/users/dfa425e42202421c9e1c8564456c67f7 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:14:07.446756 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915da249-fdad-4b9b-8b51-fbda844e0eed service neutron] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:14:07.462939 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915da249-fdad-4b9b-8b51-fbda844e0eed service neutron] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:14:07.529859 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3eb900-6378-4808-9d1e-d8e1e8f4888d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:14:07.530729 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3eb900-6378-4808-9d1e-d8e1e8f4888d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:14:07.531014 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3eb900-6378-4808-9d1e-d8e1e8f4888d admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:14:07.531289 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3eb900-6378-4808-9d1e-d8e1e8f4888d admin admin] PATH_INFO: `/v3/projects/d58410e922bd46418c9e93bf61b348f8` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:14:07.532470 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3eb900-6378-4808-9d1e-d8e1e8f4888d admin admin] RBAC: Authorizing `identity:delete_project(project_id=d58410e922bd46418c9e93bf61b348f8)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:14:07.534414 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3eb900-6378-4808-9d1e-d8e1e8f4888d admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:14:07.584189 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-8d3eb900-6378-4808-9d1e-d8e1e8f4888d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd58410e922bd46418c9e93bf61b348f8', 'request_id': 'req-8d3eb900-6378-4808-9d1e-d8e1e8f4888d'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:14:07.593156 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-8d3eb900-6378-4808-9d1e-d8e1e8f4888d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd58410e922bd46418c9e93bf61b348f8', 'request_id': 'req-8d3eb900-6378-4808-9d1e-d8e1e8f4888d'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:14:07.598818 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-8d3eb900-6378-4808-9d1e-d8e1e8f4888d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd58410e922bd46418c9e93bf61b348f8', 'request_id': 'req-8d3eb900-6378-4808-9d1e-d8e1e8f4888d'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:14:07.617510 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-8d3eb900-6378-4808-9d1e-d8e1e8f4888d admin admin] The token cache is being invalidate because project d58410e922bd46418c9e93bf61b348f8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 15:14:07.617716 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-8d3eb900-6378-4808-9d1e-d8e1e8f4888d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:14:07.618365 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-8d3eb900-6378-4808-9d1e-d8e1e8f4888d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:14:07.620433 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1124/2263] 10.222.0.46 () {58 vars in 1232 bytes} [Thu May 26 15:14:07 2022] DELETE /identity/v3/projects/d58410e922bd46418c9e93bf61b348f8 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:14:11.442169 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757bed63-cf3f-428e-9610-eea0260a782d None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:14:11.442679 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757bed63-cf3f-428e-9610-eea0260a782d None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:14:11.442790 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757bed63-cf3f-428e-9610-eea0260a782d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:14:11.444528 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2442749c-2188-426f-8e68-47f45e14ad40 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:14:11.444941 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2442749c-2188-426f-8e68-47f45e14ad40 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:14:11.445401 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2442749c-2188-426f-8e68-47f45e14ad40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:14:11.461049 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-757bed63-cf3f-428e-9610-eea0260a782d None None] MFA Rules not processed for user `e95b6425286f4235b9d1802acddac4b8`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:14:11.462299 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-757bed63-cf3f-428e-9610-eea0260a782d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:14:11.463752 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-2442749c-2188-426f-8e68-47f45e14ad40 None None] MFA Rules not processed for user `27172e53a12a437c9d57a75a95727d71`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:14:11.464980 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-2442749c-2188-426f-8e68-47f45e14ad40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:14:11.521905 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1125/2264] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:14:11 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:14:11.522369 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1140/2265] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:14:11 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 15:14:11.820412 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2442749c-2188-426f-8e68-47f45e14ad40 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:14:11.821977 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-2442749c-2188-426f-8e68-47f45e14ad40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:14:11.866688 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:14:11.887035 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2442749c-2188-426f-8e68-47f45e14ad40 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:14:11.891083 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3867d49-44f0-420a-8147-9d410bae3fa3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0156a2f6f3794f479e76b85bdbbe5c26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:14:11.891731 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3867d49-44f0-420a-8147-9d410bae3fa3 service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:14:11.892033 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3867d49-44f0-420a-8147-9d410bae3fa3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:14:11.892289 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3867d49-44f0-420a-8147-9d410bae3fa3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:14:11.894322 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-a3867d49-44f0-420a-8147-9d410bae3fa3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:14:11.904141 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3867d49-44f0-420a-8147-9d410bae3fa3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:14:11.906206 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3867d49-44f0-420a-8147-9d410bae3fa3 service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:14:11.914698 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1126/2266] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:14:11 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 15:14:12.257867 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757bed63-cf3f-428e-9610-eea0260a782d None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:14:12.259254 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-757bed63-cf3f-428e-9610-eea0260a782d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:14:12.297586 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:14:12.345465 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757bed63-cf3f-428e-9610-eea0260a782d None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:14:12.350658 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9c84cb-ab2c-4156-a3eb-b31bc72bb253 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf55a00e3b084aa3b22c25a90e6a773c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:14:12.351344 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9c84cb-ab2c-4156-a3eb-b31bc72bb253 service placement] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:14:12.351631 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9c84cb-ab2c-4156-a3eb-b31bc72bb253 service placement] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:14:12.351909 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9c84cb-ab2c-4156-a3eb-b31bc72bb253 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:14:12.353756 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-ed9c84cb-ab2c-4156-a3eb-b31bc72bb253 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:14:12.365807 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9c84cb-ab2c-4156-a3eb-b31bc72bb253 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:14:12.367407 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9c84cb-ab2c-4156-a3eb-b31bc72bb253 service placement] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:14:12.379340 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1141/2267] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:14:12 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 15:14:14.415237 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3867d49-44f0-420a-8147-9d410bae3fa3 service neutron] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:14:14.416632 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-a3867d49-44f0-420a-8147-9d410bae3fa3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:14:14.491925 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3867d49-44f0-420a-8147-9d410bae3fa3 service neutron] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:14:14.497654 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177de24e-f3f5-46e6-a017-4bddff6f54ed service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6639353c70449d1b3d10cd6487c3b00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:14:14.498303 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177de24e-f3f5-46e6-a017-4bddff6f54ed service nova] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:14:14.498588 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177de24e-f3f5-46e6-a017-4bddff6f54ed service nova] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:14:14.498867 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177de24e-f3f5-46e6-a017-4bddff6f54ed service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:14:14.500660 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-177de24e-f3f5-46e6-a017-4bddff6f54ed service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:14:14.511670 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177de24e-f3f5-46e6-a017-4bddff6f54ed service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:14:14.513268 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177de24e-f3f5-46e6-a017-4bddff6f54ed service nova] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:14:14.522244 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1127/2268] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:14:14 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 15:14:18.666317 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9c84cb-ab2c-4156-a3eb-b31bc72bb253 service placement] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:14:18.679735 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9c84cb-ab2c-4156-a3eb-b31bc72bb253 service placement] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:14:18.685838 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503d73a3-906a-4711-b2d4-250c367a80c3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6639353c70449d1b3d10cd6487c3b00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:14:18.687271 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503d73a3-906a-4711-b2d4-250c367a80c3 service nova] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:14:18.687646 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503d73a3-906a-4711-b2d4-250c367a80c3 service nova] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:14:18.687931 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503d73a3-906a-4711-b2d4-250c367a80c3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:14:18.690056 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-503d73a3-906a-4711-b2d4-250c367a80c3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:14:18.703334 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503d73a3-906a-4711-b2d4-250c367a80c3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:14:18.704996 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503d73a3-906a-4711-b2d4-250c367a80c3 service nova] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:14:18.716150 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1142/2269] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:14:18 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 15:14:21.024204 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177de24e-f3f5-46e6-a017-4bddff6f54ed service nova] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:14:21.025760 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-177de24e-f3f5-46e6-a017-4bddff6f54ed service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:14:21.041044 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177de24e-f3f5-46e6-a017-4bddff6f54ed service nova] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:14:21.046659 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704909b0-7408-4272-8dbe-0cc6fb03ac21 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6639353c70449d1b3d10cd6487c3b00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:14:21.047304 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704909b0-7408-4272-8dbe-0cc6fb03ac21 service nova] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:14:21.047587 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704909b0-7408-4272-8dbe-0cc6fb03ac21 service nova] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:14:21.047862 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704909b0-7408-4272-8dbe-0cc6fb03ac21 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:14:21.049893 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-704909b0-7408-4272-8dbe-0cc6fb03ac21 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:14:21.113048 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704909b0-7408-4272-8dbe-0cc6fb03ac21 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:14:21.114723 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704909b0-7408-4272-8dbe-0cc6fb03ac21 service nova] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:14:21.124923 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1128/2270] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:14:21 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 15:14:21.993155 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503d73a3-906a-4711-b2d4-250c367a80c3 service nova] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:14:22.005288 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503d73a3-906a-4711-b2d4-250c367a80c3 service nova] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:14:22.011151 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830a8bf6-82dd-421a-a6b5-26bd415b89de service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6639353c70449d1b3d10cd6487c3b00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:14:22.011795 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830a8bf6-82dd-421a-a6b5-26bd415b89de service nova] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:14:22.012079 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830a8bf6-82dd-421a-a6b5-26bd415b89de service nova] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:14:22.012353 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830a8bf6-82dd-421a-a6b5-26bd415b89de service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:14:22.014453 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-830a8bf6-82dd-421a-a6b5-26bd415b89de service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:14:22.077543 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830a8bf6-82dd-421a-a6b5-26bd415b89de service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:14:22.079152 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830a8bf6-82dd-421a-a6b5-26bd415b89de service nova] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:14:22.088534 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1143/2271] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:14:21 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 15:14:31.120105 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=72 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:14:32.084055 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=55 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:14:39.364490 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cede3b1-f286-492d-9b15-c4d622a22e29 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:14:39.365092 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cede3b1-f286-492d-9b15-c4d622a22e29 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:14:39.365419 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cede3b1-f286-492d-9b15-c4d622a22e29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:14:39.384984 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-8cede3b1-f286-492d-9b15-c4d622a22e29 None None] MFA Rules not processed for user `2ce5b5d5a12743bc80a4b191c24ff52f`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:14:39.386235 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-8cede3b1-f286-492d-9b15-c4d622a22e29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:14:39.393993 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1129/2272] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:14:39 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:14:39.719751 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830a8bf6-82dd-421a-a6b5-26bd415b89de service nova] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:14:39.721138 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-830a8bf6-82dd-421a-a6b5-26bd415b89de service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:14:39.736244 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830a8bf6-82dd-421a-a6b5-26bd415b89de service nova] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:14:39.739990 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf3e3a6-4031-4f38-b5c9-c7e300dcbe02 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0156a2f6f3794f479e76b85bdbbe5c26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:14:39.740644 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf3e3a6-4031-4f38-b5c9-c7e300dcbe02 service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:14:39.740929 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf3e3a6-4031-4f38-b5c9-c7e300dcbe02 service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:14:39.741210 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf3e3a6-4031-4f38-b5c9-c7e300dcbe02 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:14:39.743174 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-5bf3e3a6-4031-4f38-b5c9-c7e300dcbe02 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:14:39.753983 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf3e3a6-4031-4f38-b5c9-c7e300dcbe02 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:14:39.755418 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf3e3a6-4031-4f38-b5c9-c7e300dcbe02 service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:14:39.764044 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1144/2273] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:14:39 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 15:14:42.412614 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353df557-ee43-4077-b3e6-7d3f458c5b42 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:14:42.413486 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353df557-ee43-4077-b3e6-7d3f458c5b42 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:14:42.413486 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353df557-ee43-4077-b3e6-7d3f458c5b42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:14:42.440642 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-353df557-ee43-4077-b3e6-7d3f458c5b42 None None] MFA Rules not processed for user `8028c520d93449a6a087c281280ee13d`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:14:42.443182 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-353df557-ee43-4077-b3e6-7d3f458c5b42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:14:42.518361 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1130/2274] 10.222.0.46 () {58 vars in 1009 bytes} [Thu May 26 15:14:42 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:14:42.578895 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf3e3a6-4031-4f38-b5c9-c7e300dcbe02 service neutron] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:14:42.580416 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-5bf3e3a6-4031-4f38-b5c9-c7e300dcbe02 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:14:42.695979 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf3e3a6-4031-4f38-b5c9-c7e300dcbe02 service neutron] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:14:42.701926 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cdb0e5-ea57-4091-95a6-ab0748630096 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4bea272624b45d8ac2ad27bd07df885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:14:42.702895 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cdb0e5-ea57-4091-95a6-ab0748630096 service swift] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:14:42.703353 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cdb0e5-ea57-4091-95a6-ab0748630096 service swift] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:14:42.703783 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cdb0e5-ea57-4091-95a6-ab0748630096 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:14:42.717010 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20cdb0e5-ea57-4091-95a6-ab0748630096 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:14:42.719704 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20cdb0e5-ea57-4091-95a6-ab0748630096 service swift] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:14:42.731311 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1145/2275] 10.222.0.46 () {60 vars in 1384 bytes} [Thu May 26 15:14:42 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 26 15:14:43.793616 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353df557-ee43-4077-b3e6-7d3f458c5b42 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:14:43.831980 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353df557-ee43-4077-b3e6-7d3f458c5b42 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:14:43.842207 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18168709-aec3-4e44-9c02-461af527a5c2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6639353c70449d1b3d10cd6487c3b00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:14:43.843838 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18168709-aec3-4e44-9c02-461af527a5c2 service nova] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:14:43.844292 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18168709-aec3-4e44-9c02-461af527a5c2 service nova] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:14:43.845878 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18168709-aec3-4e44-9c02-461af527a5c2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:14:43.848042 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-18168709-aec3-4e44-9c02-461af527a5c2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:14:43.868119 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18168709-aec3-4e44-9c02-461af527a5c2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:14:43.877283 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18168709-aec3-4e44-9c02-461af527a5c2 service nova] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:14:43.910309 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1131/2276] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:14:43 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 15:14:52.726823 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=38 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:14:53.900623 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=34 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:14:56.982324 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a9a9a2-ea4a-4e1c-9631-6aa1413a7b73 None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:14:56.983444 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a9a9a2-ea4a-4e1c-9631-6aa1413a7b73 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:14:56.983559 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a9a9a2-ea4a-4e1c-9631-6aa1413a7b73 None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:14:56.985141 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1146/2277] 10.222.0.46 () {54 vars in 921 bytes} [Thu May 26 15:14:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 15:14:56.992704 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca6ca67-fb4a-4762-96f1-5a7f0c6f759a None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:14:56.993538 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca6ca67-fb4a-4762-96f1-5a7f0c6f759a None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:14:56.994146 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca6ca67-fb4a-4762-96f1-5a7f0c6f759a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:14:57.018891 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-7ca6ca67-fb4a-4762-96f1-5a7f0c6f759a None None] MFA Rules not processed for user `d6639353c70449d1b3d10cd6487c3b00`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:14:57.020647 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-7ca6ca67-fb4a-4762-96f1-5a7f0c6f759a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:14:57.033873 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1132/2278] 10.222.0.46 () {58 vars in 1018 bytes} [Thu May 26 15:14:56 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:14:57.048368 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a9a9a2-ea4a-4e1c-9631-6aa1413a7b73 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:14:57.050652 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-05a9a9a2-ea4a-4e1c-9631-6aa1413a7b73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:14:57.153020 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a9a9a2-ea4a-4e1c-9631-6aa1413a7b73 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:14:57.158072 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a08e6d0-85d8-47de-ae37-11e1612c0a63 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a436dbdee624f44ad76cca8416b0ad1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:14:57.158845 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a08e6d0-85d8-47de-ae37-11e1612c0a63 service glance] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:14:57.159298 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a08e6d0-85d8-47de-ae37-11e1612c0a63 service glance] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:14:57.159577 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a08e6d0-85d8-47de-ae37-11e1612c0a63 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:14:57.170714 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a08e6d0-85d8-47de-ae37-11e1612c0a63 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:14:57.172214 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a08e6d0-85d8-47de-ae37-11e1612c0a63 service glance] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:14:57.183078 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1147/2279] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:14:57 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 15:14:57.257433 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e260f6-5b6b-4fd4-9bb0-c2225e57ae83 None None] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:14:57.257847 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e260f6-5b6b-4fd4-9bb0-c2225e57ae83 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:14:57.258132 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e260f6-5b6b-4fd4-9bb0-c2225e57ae83 None None] PATH_INFO: `/` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:14:57.259509 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1133/2280] 10.222.0.46 () {54 vars in 921 bytes} [Thu May 26 15:14:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 15:14:57.265341 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae02af5-2eca-4c39-b51b-dc980dc28aa8 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:14:57.265657 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae02af5-2eca-4c39-b51b-dc980dc28aa8 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:14:57.265915 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae02af5-2eca-4c39-b51b-dc980dc28aa8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:14:57.285059 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-6ae02af5-2eca-4c39-b51b-dc980dc28aa8 None None] MFA Rules not processed for user `0156a2f6f3794f479e76b85bdbbe5c26`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:14:57.286300 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-6ae02af5-2eca-4c39-b51b-dc980dc28aa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:14:57.293155 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1148/2281] 10.222.0.46 () {58 vars in 1018 bytes} [Thu May 26 15:14:57 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:14:57.307324 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e260f6-5b6b-4fd4-9bb0-c2225e57ae83 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:14:57.319238 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e260f6-5b6b-4fd4-9bb0-c2225e57ae83 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:14:57.323159 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1624b4-9468-427a-a4f3-51a8f8bc9fa0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0156a2f6f3794f479e76b85bdbbe5c26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:14:57.323882 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1624b4-9468-427a-a4f3-51a8f8bc9fa0 service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:14:57.324168 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1624b4-9468-427a-a4f3-51a8f8bc9fa0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:14:57.324580 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1624b4-9468-427a-a4f3-51a8f8bc9fa0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:14:57.333646 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1624b4-9468-427a-a4f3-51a8f8bc9fa0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:14:57.335161 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1624b4-9468-427a-a4f3-51a8f8bc9fa0 service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:14:57.343865 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1134/2282] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:14:57 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 15:14:57.427207 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b16be3-53d4-47aa-a548-12d9cb53499b None None] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:14:57.427500 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b16be3-53d4-47aa-a548-12d9cb53499b None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:14:57.427833 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b16be3-53d4-47aa-a548-12d9cb53499b None None] PATH_INFO: `/` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:14:57.429203 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1149/2283] 10.222.0.46 () {54 vars in 921 bytes} [Thu May 26 15:14:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 26 15:14:57.435111 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bbc2c2-f170-4ff6-bc3a-a88a7138e04b None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:14:57.435394 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bbc2c2-f170-4ff6-bc3a-a88a7138e04b None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:14:57.435670 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bbc2c2-f170-4ff6-bc3a-a88a7138e04b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:14:57.454180 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-c9bbc2c2-f170-4ff6-bc3a-a88a7138e04b None None] MFA Rules not processed for user `d6639353c70449d1b3d10cd6487c3b00`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:14:57.455293 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-c9bbc2c2-f170-4ff6-bc3a-a88a7138e04b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:14:57.464337 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1135/2284] 10.222.0.46 () {58 vars in 1018 bytes} [Thu May 26 15:14:57 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:14:57.475773 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b16be3-53d4-47aa-a548-12d9cb53499b None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:14:57.487657 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b16be3-53d4-47aa-a548-12d9cb53499b None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:14:57.491407 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e646ab0c-7d46-4ad3-ae05-8958cc32878c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0156a2f6f3794f479e76b85bdbbe5c26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:14:57.492071 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e646ab0c-7d46-4ad3-ae05-8958cc32878c service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:14:57.492355 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e646ab0c-7d46-4ad3-ae05-8958cc32878c service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:14:57.492634 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e646ab0c-7d46-4ad3-ae05-8958cc32878c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:14:57.505015 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e646ab0c-7d46-4ad3-ae05-8958cc32878c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:14:57.506497 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e646ab0c-7d46-4ad3-ae05-8958cc32878c service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:14:57.517875 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1150/2285] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:14:57 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 15:15:03.101866 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dec67db-4111-469d-9fdb-a507a5348cd7 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:15:03.103068 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dec67db-4111-469d-9fdb-a507a5348cd7 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:15:03.103356 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dec67db-4111-469d-9fdb-a507a5348cd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:15:03.122390 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-0dec67db-4111-469d-9fdb-a507a5348cd7 None None] MFA Rules not processed for user `8028c520d93449a6a087c281280ee13d`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:15:03.124138 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-0dec67db-4111-469d-9fdb-a507a5348cd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:15:03.132209 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1136/2286] 10.222.0.46 () {58 vars in 1009 bytes} [Thu May 26 15:15:03 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:15:03.145584 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e646ab0c-7d46-4ad3-ae05-8958cc32878c service neutron] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:15:03.158068 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e646ab0c-7d46-4ad3-ae05-8958cc32878c service neutron] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:15:03.164348 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515a1569-4a11-4ff1-a2e5-d3c60c4b6bc1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4bea272624b45d8ac2ad27bd07df885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:15:03.165241 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515a1569-4a11-4ff1-a2e5-d3c60c4b6bc1 service swift] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:15:03.165707 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515a1569-4a11-4ff1-a2e5-d3c60c4b6bc1 service swift] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:15:03.165995 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515a1569-4a11-4ff1-a2e5-d3c60c4b6bc1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:15:03.176388 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515a1569-4a11-4ff1-a2e5-d3c60c4b6bc1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:15:03.178237 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515a1569-4a11-4ff1-a2e5-d3c60c4b6bc1 service swift] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:15:03.188542 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1151/2287] 10.222.0.46 () {60 vars in 1384 bytes} [Thu May 26 15:15:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 26 15:15:09.662941 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dec67db-4111-469d-9fdb-a507a5348cd7 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:15:09.674052 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dec67db-4111-469d-9fdb-a507a5348cd7 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:15:09.678116 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4168898a-885e-452b-90f6-607990b619c6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0156a2f6f3794f479e76b85bdbbe5c26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:15:09.679068 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4168898a-885e-452b-90f6-607990b619c6 service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:15:09.679569 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4168898a-885e-452b-90f6-607990b619c6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:15:09.680067 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4168898a-885e-452b-90f6-607990b619c6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:15:09.682410 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-4168898a-885e-452b-90f6-607990b619c6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:15:09.694498 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4168898a-885e-452b-90f6-607990b619c6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:15:09.695945 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4168898a-885e-452b-90f6-607990b619c6 service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:15:09.704990 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1137/2288] 10.222.0.46 () {60 vars in 1396 bytes} [Thu May 26 15:15:09 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3260 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 15:15:13.184150 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=50 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:15:19.700727 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=21 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:15:21.915103 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccb8428-7c04-4d83-ac76-e70b131e05ae None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:15:21.915607 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccb8428-7c04-4d83-ac76-e70b131e05ae None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:15:21.915694 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccb8428-7c04-4d83-ac76-e70b131e05ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:15:21.933764 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-2ccb8428-7c04-4d83-ac76-e70b131e05ae None None] MFA Rules not processed for user `8028c520d93449a6a087c281280ee13d`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:15:21.935030 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-2ccb8428-7c04-4d83-ac76-e70b131e05ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:15:21.942998 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1152/2289] 10.222.0.46 () {58 vars in 967 bytes} [Thu May 26 15:15:21 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:15:21.959081 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4168898a-885e-452b-90f6-607990b619c6 service neutron] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:15:21.972609 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4168898a-885e-452b-90f6-607990b619c6 service neutron] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:15:21.977738 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd3ce0a-1528-4f24-a8df-500b43eea101 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4bea272624b45d8ac2ad27bd07df885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:15:21.978422 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd3ce0a-1528-4f24-a8df-500b43eea101 service swift] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:15:21.978709 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd3ce0a-1528-4f24-a8df-500b43eea101 service swift] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:15:21.978992 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd3ce0a-1528-4f24-a8df-500b43eea101 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:15:21.987676 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd3ce0a-1528-4f24-a8df-500b43eea101 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:15:21.989360 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd3ce0a-1528-4f24-a8df-500b43eea101 service swift] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:15:21.999262 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1138/2290] 10.222.0.46 () {60 vars in 1384 bytes} [Thu May 26 15:15:21 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 26 15:15:23.990017 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccb8428-7c04-4d83-ac76-e70b131e05ae None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:15:24.003677 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccb8428-7c04-4d83-ac76-e70b131e05ae None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:15:24.004141 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9feecf89-dce2-49e6-89dc-9d754f796ffe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0156a2f6f3794f479e76b85bdbbe5c26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:15:24.004625 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9feecf89-dce2-49e6-89dc-9d754f796ffe service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:15:24.004907 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9feecf89-dce2-49e6-89dc-9d754f796ffe service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:15:24.005184 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9feecf89-dce2-49e6-89dc-9d754f796ffe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:15:24.007040 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-9feecf89-dce2-49e6-89dc-9d754f796ffe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:15:24.071811 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9feecf89-dce2-49e6-89dc-9d754f796ffe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:15:24.073246 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9feecf89-dce2-49e6-89dc-9d754f796ffe service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:15:24.087313 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1153/2291] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:15:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 15:15:27.997648 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd3ce0a-1528-4f24-a8df-500b43eea101 service swift] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:15:28.023684 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd3ce0a-1528-4f24-a8df-500b43eea101 service swift] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:15:28.033425 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e899cde-27a0-4e5a-aab1-8215d24c6d6a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf55a00e3b084aa3b22c25a90e6a773c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:15:28.034804 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e899cde-27a0-4e5a-aab1-8215d24c6d6a service placement] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:15:28.036238 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e899cde-27a0-4e5a-aab1-8215d24c6d6a service placement] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:15:28.037027 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e899cde-27a0-4e5a-aab1-8215d24c6d6a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:15:28.040033 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-3e899cde-27a0-4e5a-aab1-8215d24c6d6a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:15:28.059658 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e899cde-27a0-4e5a-aab1-8215d24c6d6a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:15:28.061307 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e899cde-27a0-4e5a-aab1-8215d24c6d6a service placement] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:15:28.076641 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1139/2292] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:15:27 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 15:15:28.092918 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9feecf89-dce2-49e6-89dc-9d754f796ffe service neutron] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:15:28.095636 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-9feecf89-dce2-49e6-89dc-9d754f796ffe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:15:28.109655 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9feecf89-dce2-49e6-89dc-9d754f796ffe service neutron] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:15:28.115297 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb995e7-045d-43e0-b18a-a7a045c35f2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:15:28.116194 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb995e7-045d-43e0-b18a-a7a045c35f2a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:15:28.116480 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb995e7-045d-43e0-b18a-a7a045c35f2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:15:28.116764 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb995e7-045d-43e0-b18a-a7a045c35f2a admin admin] PATH_INFO: `/v3/users/27172e53a12a437c9d57a75a95727d71` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:15:28.118318 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb995e7-045d-43e0-b18a-a7a045c35f2a admin admin] RBAC: Authorizing `identity:delete_user(user_id=27172e53a12a437c9d57a75a95727d71)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:15:28.120378 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb995e7-045d-43e0-b18a-a7a045c35f2a admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:15:28.177863 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-2fb995e7-045d-43e0-b18a-a7a045c35f2a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '27172e53a12a437c9d57a75a95727d71'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:15:28.187158 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-2fb995e7-045d-43e0-b18a-a7a045c35f2a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '27172e53a12a437c9d57a75a95727d71'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:15:28.188390 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-2fb995e7-045d-43e0-b18a-a7a045c35f2a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '27172e53a12a437c9d57a75a95727d71'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:15:28.208566 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-2fb995e7-045d-43e0-b18a-a7a045c35f2a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '27172e53a12a437c9d57a75a95727d71'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:15:28.219997 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-2fb995e7-045d-43e0-b18a-a7a045c35f2a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '27172e53a12a437c9d57a75a95727d71'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:15:28.226781 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1154/2293] 10.222.0.46 () {58 vars in 1223 bytes} [Thu May 26 15:15:28 2022] DELETE /identity/v3/users/27172e53a12a437c9d57a75a95727d71 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:15:28.443803 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e899cde-27a0-4e5a-aab1-8215d24c6d6a service placement] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:15:28.461736 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e899cde-27a0-4e5a-aab1-8215d24c6d6a service placement] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:15:28.531646 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6035335-4176-4f42-a529-93f9677d7ba2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:15:28.532422 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6035335-4176-4f42-a529-93f9677d7ba2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:15:28.532701 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6035335-4176-4f42-a529-93f9677d7ba2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:15:28.532970 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6035335-4176-4f42-a529-93f9677d7ba2 admin admin] PATH_INFO: `/v3/projects/45e3fe61cc704411ac645e5935b9d258` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:15:28.534309 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6035335-4176-4f42-a529-93f9677d7ba2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=45e3fe61cc704411ac645e5935b9d258)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:15:28.536342 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6035335-4176-4f42-a529-93f9677d7ba2 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:15:28.599921 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-f6035335-4176-4f42-a529-93f9677d7ba2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '45e3fe61cc704411ac645e5935b9d258', 'request_id': 'req-f6035335-4176-4f42-a529-93f9677d7ba2'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:15:28.612637 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-f6035335-4176-4f42-a529-93f9677d7ba2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '45e3fe61cc704411ac645e5935b9d258', 'request_id': 'req-f6035335-4176-4f42-a529-93f9677d7ba2'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:15:28.620011 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-f6035335-4176-4f42-a529-93f9677d7ba2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '45e3fe61cc704411ac645e5935b9d258', 'request_id': 'req-f6035335-4176-4f42-a529-93f9677d7ba2'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:15:28.634008 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-f6035335-4176-4f42-a529-93f9677d7ba2 admin admin] The token cache is being invalidate because project 45e3fe61cc704411ac645e5935b9d258 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 15:15:28.634303 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-f6035335-4176-4f42-a529-93f9677d7ba2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:15:28.634949 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-f6035335-4176-4f42-a529-93f9677d7ba2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:15:28.636516 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1140/2294] 10.222.0.46 () {58 vars in 1232 bytes} [Thu May 26 15:15:28 2022] DELETE /identity/v3/projects/45e3fe61cc704411ac645e5935b9d258 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 15:15:28.642159 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b6beeb-5af2-4336-b293-b5080c722a1c None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:15:28.642434 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b6beeb-5af2-4336-b293-b5080c722a1c None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:15:28.642716 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b6beeb-5af2-4336-b293-b5080c722a1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:15:28.660915 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-28b6beeb-5af2-4336-b293-b5080c722a1c None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:15:28.662176 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-28b6beeb-5af2-4336-b293-b5080c722a1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:15:28.719495 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:15:28.732159 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1155/2295] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:15:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:15:28.739986 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2c79fb-f7d7-45be-97c8-6be0b0817e8d None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:15:28.740299 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2c79fb-f7d7-45be-97c8-6be0b0817e8d None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:15:28.740580 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2c79fb-f7d7-45be-97c8-6be0b0817e8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:15:28.760632 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-4f2c79fb-f7d7-45be-97c8-6be0b0817e8d None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:15:28.761910 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-4f2c79fb-f7d7-45be-97c8-6be0b0817e8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:15:28.769985 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1141/2296] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:15:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:15:28.774216 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b6beeb-5af2-4336-b293-b5080c722a1c None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:15:28.784362 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b6beeb-5af2-4336-b293-b5080c722a1c None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:15:28.789226 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3340377-19ea-47dc-aa8c-1bd128f9bc7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:15:28.789950 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3340377-19ea-47dc-aa8c-1bd128f9bc7d admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:15:28.790228 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3340377-19ea-47dc-aa8c-1bd128f9bc7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:15:28.790503 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3340377-19ea-47dc-aa8c-1bd128f9bc7d admin admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:15:28.791235 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3340377-19ea-47dc-aa8c-1bd128f9bc7d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 15:15:28.791637 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3340377-19ea-47dc-aa8c-1bd128f9bc7d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:15:28.793138 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3340377-19ea-47dc-aa8c-1bd128f9bc7d admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:15:28.808523 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1156/2297] 10.222.0.46 () {58 vars in 1152 bytes} [Thu May 26 15:15:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 15:15:28.814769 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2c79fb-f7d7-45be-97c8-6be0b0817e8d None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:15:28.824320 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2c79fb-f7d7-45be-97c8-6be0b0817e8d None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:15:28.829279 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67cccba-7317-486e-8941-41a98e285d0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:15:28.829973 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67cccba-7317-486e-8941-41a98e285d0b admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:15:28.830316 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67cccba-7317-486e-8941-41a98e285d0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:15:28.830602 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67cccba-7317-486e-8941-41a98e285d0b admin admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:15:28.831309 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67cccba-7317-486e-8941-41a98e285d0b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:15:28.833238 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67cccba-7317-486e-8941-41a98e285d0b admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:15:28.871820 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1142/2298] 10.222.0.46 () {60 vars in 1152 bytes} [Thu May 26 15:15:28 2022] POST /identity/v3/projects => generated 383 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 15:15:28.876610 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3340377-19ea-47dc-aa8c-1bd128f9bc7d admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:15:28.886805 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3340377-19ea-47dc-aa8c-1bd128f9bc7d admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:15:28.944307 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72013d8d-040a-4c38-9dc3-852d85fe621d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:15:28.944986 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72013d8d-040a-4c38-9dc3-852d85fe621d admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:15:28.945272 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72013d8d-040a-4c38-9dc3-852d85fe621d admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:15:28.945578 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72013d8d-040a-4c38-9dc3-852d85fe621d admin admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:15:28.946279 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72013d8d-040a-4c38-9dc3-852d85fe621d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:15:28.948244 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72013d8d-040a-4c38-9dc3-852d85fe621d admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:15:28.973303 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1157/2299] 10.222.0.46 () {60 vars in 1143 bytes} [Thu May 26 15:15:28 2022] POST /identity/v3/users => generated 292 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 15:15:28.977296 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67cccba-7317-486e-8941-41a98e285d0b admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:15:28.988520 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67cccba-7317-486e-8941-41a98e285d0b admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:15:28.993731 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79e8502-34dd-49b3-8643-6dd50a843608 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:15:28.994406 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79e8502-34dd-49b3-8643-6dd50a843608 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:15:28.994691 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79e8502-34dd-49b3-8643-6dd50a843608 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:15:28.994971 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79e8502-34dd-49b3-8643-6dd50a843608 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:15:28.995704 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79e8502-34dd-49b3-8643-6dd50a843608 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:15:28.997955 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c79e8502-34dd-49b3-8643-6dd50a843608 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:15:29.009267 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1143/2300] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 15:15:28 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 15:15:29.013200 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72013d8d-040a-4c38-9dc3-852d85fe621d admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:15:29.023526 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72013d8d-040a-4c38-9dc3-852d85fe621d admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:15:29.028768 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ff4b60-dde2-4867-b786-364570c853df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:15:29.029439 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ff4b60-dde2-4867-b786-364570c853df admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:15:29.029754 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ff4b60-dde2-4867-b786-364570c853df admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:15:29.030360 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ff4b60-dde2-4867-b786-364570c853df admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:15:29.030777 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ff4b60-dde2-4867-b786-364570c853df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:15:29.032550 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ff4b60-dde2-4867-b786-364570c853df admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:15:29.043154 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1158/2301] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 15:15:29 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 15:15:29.048021 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79e8502-34dd-49b3-8643-6dd50a843608 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:15:29.059677 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79e8502-34dd-49b3-8643-6dd50a843608 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:15:29.067313 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40287af-7180-4ce5-9f6e-b66986eaa36b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:15:29.068253 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40287af-7180-4ce5-9f6e-b66986eaa36b admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:15:29.068539 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40287af-7180-4ce5-9f6e-b66986eaa36b admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:15:29.068817 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40287af-7180-4ce5-9f6e-b66986eaa36b admin admin] PATH_INFO: `/v3/projects/5d165a22a1a742028af84f2e00d2176f/users/83aba5a715d84a199ca388b65139c200/roles/d4088d1845a24fbd9bd8975ce0b1d3d0` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:15:29.082983 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:15:29.087222 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40287af-7180-4ce5-9f6e-b66986eaa36b admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d165a22a1a742028af84f2e00d2176f, user_id=83aba5a715d84a199ca388b65139c200, role_id=d4088d1845a24fbd9bd8975ce0b1d3d0)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:15:29.090944 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40287af-7180-4ce5-9f6e-b66986eaa36b admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:15:29.114397 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1144/2302] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 15:15:29 2022] PUT /identity/v3/projects/5d165a22a1a742028af84f2e00d2176f/users/83aba5a715d84a199ca388b65139c200/roles/d4088d1845a24fbd9bd8975ce0b1d3d0 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:15:29.125971 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f792e1c0-8af8-44ad-8203-650be35c136c None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:15:29.126577 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f792e1c0-8af8-44ad-8203-650be35c136c None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:15:29.126577 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f792e1c0-8af8-44ad-8203-650be35c136c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:15:29.158837 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-f792e1c0-8af8-44ad-8203-650be35c136c None None] MFA Rules not processed for user `83aba5a715d84a199ca388b65139c200`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:15:29.160066 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-f792e1c0-8af8-44ad-8203-650be35c136c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:15:29.226826 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1159/2303] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:15:29 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:15:29.232902 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40287af-7180-4ce5-9f6e-b66986eaa36b admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:15:29.301334 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40287af-7180-4ce5-9f6e-b66986eaa36b admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:15:29.306253 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598a3aa4-d8aa-440c-baad-6378b066b1e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:15:29.306931 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598a3aa4-d8aa-440c-baad-6378b066b1e9 admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:15:29.307220 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598a3aa4-d8aa-440c-baad-6378b066b1e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:15:29.307506 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598a3aa4-d8aa-440c-baad-6378b066b1e9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:15:29.308223 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598a3aa4-d8aa-440c-baad-6378b066b1e9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:15:29.310270 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598a3aa4-d8aa-440c-baad-6378b066b1e9 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:15:29.337337 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1145/2304] 10.222.0.46 () {60 vars in 1152 bytes} [Thu May 26 15:15:29 2022] POST /identity/v3/projects => generated 387 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 15:15:29.342165 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f792e1c0-8af8-44ad-8203-650be35c136c None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:15:29.353536 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f792e1c0-8af8-44ad-8203-650be35c136c None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:15:29.409578 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:15:29.418482 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f19fbb0-0a71-45c1-bbc1-a53bb93e6262 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:15:29.419236 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f19fbb0-0a71-45c1-bbc1-a53bb93e6262 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:15:29.419527 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f19fbb0-0a71-45c1-bbc1-a53bb93e6262 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:15:29.419813 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f19fbb0-0a71-45c1-bbc1-a53bb93e6262 admin admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:15:29.420521 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f19fbb0-0a71-45c1-bbc1-a53bb93e6262 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:15:29.422536 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f19fbb0-0a71-45c1-bbc1-a53bb93e6262 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:15:29.445098 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1160/2305] 10.222.0.46 () {60 vars in 1143 bytes} [Thu May 26 15:15:29 2022] POST /identity/v3/users => generated 300 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 15:15:29.450509 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598a3aa4-d8aa-440c-baad-6378b066b1e9 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:15:29.461253 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598a3aa4-d8aa-440c-baad-6378b066b1e9 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:15:29.466611 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b6850e-37da-4cd8-9dd5-9ff5c70c0ade admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:15:29.467288 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b6850e-37da-4cd8-9dd5-9ff5c70c0ade admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:15:29.467570 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b6850e-37da-4cd8-9dd5-9ff5c70c0ade admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:15:29.467847 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78b6850e-37da-4cd8-9dd5-9ff5c70c0ade admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:15:29.468592 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b6850e-37da-4cd8-9dd5-9ff5c70c0ade admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:15:29.470351 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78b6850e-37da-4cd8-9dd5-9ff5c70c0ade admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:15:29.483133 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1146/2306] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 15:15:29 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 15:15:29.489232 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f19fbb0-0a71-45c1-bbc1-a53bb93e6262 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:15:29.506897 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f19fbb0-0a71-45c1-bbc1-a53bb93e6262 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:15:29.512918 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94642573-5947-4c72-8d68-71d32ea5b691 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:15:29.513960 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94642573-5947-4c72-8d68-71d32ea5b691 admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:15:29.514367 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94642573-5947-4c72-8d68-71d32ea5b691 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:15:29.514737 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94642573-5947-4c72-8d68-71d32ea5b691 admin admin] PATH_INFO: `/v3/projects/b038d8ff190a4599858bc63a0651d6d8/users/3918887911cd4b6996cc2ac592b8f096/roles/a2e551773cbd4df084020676865b8222` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:15:29.533384 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94642573-5947-4c72-8d68-71d32ea5b691 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b038d8ff190a4599858bc63a0651d6d8, user_id=3918887911cd4b6996cc2ac592b8f096, role_id=a2e551773cbd4df084020676865b8222)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:15:29.536383 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94642573-5947-4c72-8d68-71d32ea5b691 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:15:29.551923 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1161/2307] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 15:15:29 2022] PUT /identity/v3/projects/b038d8ff190a4599858bc63a0651d6d8/users/3918887911cd4b6996cc2ac592b8f096/roles/a2e551773cbd4df084020676865b8222 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:15:29.559026 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b6850e-37da-4cd8-9dd5-9ff5c70c0ade admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:15:29.570780 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78b6850e-37da-4cd8-9dd5-9ff5c70c0ade admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:15:29.645971 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f913be73-722e-463e-8ddc-117ebcebc5b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:15:29.646655 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f913be73-722e-463e-8ddc-117ebcebc5b3 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:15:29.646940 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f913be73-722e-463e-8ddc-117ebcebc5b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:15:29.647215 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f913be73-722e-463e-8ddc-117ebcebc5b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:15:29.647953 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f913be73-722e-463e-8ddc-117ebcebc5b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:15:29.649600 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f913be73-722e-463e-8ddc-117ebcebc5b3 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:15:29.662509 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1147/2308] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 15:15:29 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 15:15:29.666629 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94642573-5947-4c72-8d68-71d32ea5b691 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:15:29.678202 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94642573-5947-4c72-8d68-71d32ea5b691 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:15:29.684021 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fc5596-c369-4dde-898e-1c51d9f739ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:15:29.686633 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fc5596-c369-4dde-898e-1c51d9f739ff admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:15:29.686633 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fc5596-c369-4dde-898e-1c51d9f739ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:15:29.686633 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fc5596-c369-4dde-898e-1c51d9f739ff admin admin] PATH_INFO: `/v3/projects/b038d8ff190a4599858bc63a0651d6d8/users/3918887911cd4b6996cc2ac592b8f096/roles/a2e551773cbd4df084020676865b8222` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:15:29.687868 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fc5596-c369-4dde-898e-1c51d9f739ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=b038d8ff190a4599858bc63a0651d6d8, user_id=3918887911cd4b6996cc2ac592b8f096, role_id=a2e551773cbd4df084020676865b8222)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:15:29.690711 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fc5596-c369-4dde-898e-1c51d9f739ff admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:15:29.702973 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1162/2309] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 15:15:29 2022] PUT /identity/v3/projects/b038d8ff190a4599858bc63a0651d6d8/users/3918887911cd4b6996cc2ac592b8f096/roles/a2e551773cbd4df084020676865b8222 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:15:29.713542 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e48cec4-0406-49f1-b28b-b60474a33b8c None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:15:29.713838 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e48cec4-0406-49f1-b28b-b60474a33b8c None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:15:29.714120 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e48cec4-0406-49f1-b28b-b60474a33b8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:15:29.751676 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-3e48cec4-0406-49f1-b28b-b60474a33b8c None None] MFA Rules not processed for user `3918887911cd4b6996cc2ac592b8f096`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:15:29.753684 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-3e48cec4-0406-49f1-b28b-b60474a33b8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:15:29.841469 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:15:29.863225 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1148/2310] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:15:29 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:15:29.877568 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fc5596-c369-4dde-898e-1c51d9f739ff admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:15:29.879400 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-81fc5596-c369-4dde-898e-1c51d9f739ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:15:29.943643 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fc5596-c369-4dde-898e-1c51d9f739ff admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:15:29.947412 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55bd066a-be0b-4e75-ac10-48ddf84cb5de service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5b5b35614a547d783e6e77ddb8c1c03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:15:29.948072 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bd066a-be0b-4e75-ac10-48ddf84cb5de service cinder] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:15:29.948357 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bd066a-be0b-4e75-ac10-48ddf84cb5de service cinder] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:15:29.948638 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bd066a-be0b-4e75-ac10-48ddf84cb5de service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:15:29.961499 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bd066a-be0b-4e75-ac10-48ddf84cb5de service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:15:29.963031 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55bd066a-be0b-4e75-ac10-48ddf84cb5de service cinder] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:15:29.973827 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1163/2311] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:15:29 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 15:15:30.045181 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e48cec4-0406-49f1-b28b-b60474a33b8c None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:15:30.125055 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e48cec4-0406-49f1-b28b-b60474a33b8c None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:15:30.130077 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d9e62e-4653-4702-913e-5cb33aa1680a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:15:30.131078 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d9e62e-4653-4702-913e-5cb33aa1680a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:15:30.131551 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d9e62e-4653-4702-913e-5cb33aa1680a admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:15:30.131846 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d9e62e-4653-4702-913e-5cb33aa1680a admin admin] PATH_INFO: `/v3/users/83aba5a715d84a199ca388b65139c200` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:15:30.133152 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d9e62e-4653-4702-913e-5cb33aa1680a admin admin] RBAC: Authorizing `identity:delete_user(user_id=83aba5a715d84a199ca388b65139c200)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:15:30.135144 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d9e62e-4653-4702-913e-5cb33aa1680a admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:15:30.189550 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-f1d9e62e-4653-4702-913e-5cb33aa1680a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '83aba5a715d84a199ca388b65139c200'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:15:30.197756 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-f1d9e62e-4653-4702-913e-5cb33aa1680a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '83aba5a715d84a199ca388b65139c200'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:15:30.198383 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-f1d9e62e-4653-4702-913e-5cb33aa1680a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '83aba5a715d84a199ca388b65139c200'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:15:30.211159 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-f1d9e62e-4653-4702-913e-5cb33aa1680a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '83aba5a715d84a199ca388b65139c200'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:15:30.218062 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-f1d9e62e-4653-4702-913e-5cb33aa1680a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '83aba5a715d84a199ca388b65139c200'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:15:30.224109 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1149/2312] 10.222.0.46 () {58 vars in 1223 bytes} [Thu May 26 15:15:30 2022] DELETE /identity/v3/users/83aba5a715d84a199ca388b65139c200 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:15:30.228756 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ad51ae-1b6d-41be-b55f-82ca23531efe None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:15:30.229272 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ad51ae-1b6d-41be-b55f-82ca23531efe None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:15:30.229384 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ad51ae-1b6d-41be-b55f-82ca23531efe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:15:30.248309 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-76ad51ae-1b6d-41be-b55f-82ca23531efe None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:15:30.249533 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-76ad51ae-1b6d-41be-b55f-82ca23531efe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:15:30.308396 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1164/2313] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:15:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:15:30.319292 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d9e62e-4653-4702-913e-5cb33aa1680a admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:15:30.320745 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-f1d9e62e-4653-4702-913e-5cb33aa1680a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:15:30.380935 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d9e62e-4653-4702-913e-5cb33aa1680a admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:15:30.384515 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951cfcc2-da94-4111-ba4b-888fe77f5a4c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0156a2f6f3794f479e76b85bdbbe5c26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:15:30.385156 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951cfcc2-da94-4111-ba4b-888fe77f5a4c service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:15:30.385436 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951cfcc2-da94-4111-ba4b-888fe77f5a4c service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:15:30.385737 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951cfcc2-da94-4111-ba4b-888fe77f5a4c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:15:30.396746 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951cfcc2-da94-4111-ba4b-888fe77f5a4c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:15:30.398526 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951cfcc2-da94-4111-ba4b-888fe77f5a4c service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:15:30.408621 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1150/2314] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:15:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 15:15:30.710715 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ad51ae-1b6d-41be-b55f-82ca23531efe None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:15:30.712416 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-76ad51ae-1b6d-41be-b55f-82ca23531efe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:15:30.725524 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ad51ae-1b6d-41be-b55f-82ca23531efe None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:15:30.730381 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03389ade-b8ae-44da-98d1-6b0bd0f4d9ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:15:30.731928 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03389ade-b8ae-44da-98d1-6b0bd0f4d9ea admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:15:30.732208 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03389ade-b8ae-44da-98d1-6b0bd0f4d9ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:15:30.732485 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03389ade-b8ae-44da-98d1-6b0bd0f4d9ea admin admin] PATH_INFO: `/v3/projects/5d165a22a1a742028af84f2e00d2176f` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:15:30.733737 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03389ade-b8ae-44da-98d1-6b0bd0f4d9ea admin admin] RBAC: Authorizing `identity:delete_project(project_id=5d165a22a1a742028af84f2e00d2176f)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:15:30.735833 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03389ade-b8ae-44da-98d1-6b0bd0f4d9ea admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:15:30.783434 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:15:30.786650 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-03389ade-b8ae-44da-98d1-6b0bd0f4d9ea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5d165a22a1a742028af84f2e00d2176f', 'request_id': 'req-03389ade-b8ae-44da-98d1-6b0bd0f4d9ea'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:15:30.795584 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-03389ade-b8ae-44da-98d1-6b0bd0f4d9ea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5d165a22a1a742028af84f2e00d2176f', 'request_id': 'req-03389ade-b8ae-44da-98d1-6b0bd0f4d9ea'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:15:30.801832 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-03389ade-b8ae-44da-98d1-6b0bd0f4d9ea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5d165a22a1a742028af84f2e00d2176f', 'request_id': 'req-03389ade-b8ae-44da-98d1-6b0bd0f4d9ea'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:15:30.813594 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-03389ade-b8ae-44da-98d1-6b0bd0f4d9ea admin admin] The token cache is being invalidate because project 5d165a22a1a742028af84f2e00d2176f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 15:15:30.813884 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-03389ade-b8ae-44da-98d1-6b0bd0f4d9ea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:15:30.814528 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-03389ade-b8ae-44da-98d1-6b0bd0f4d9ea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:15:30.816101 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1165/2315] 10.222.0.46 () {58 vars in 1232 bytes} [Thu May 26 15:15:30 2022] DELETE /identity/v3/projects/5d165a22a1a742028af84f2e00d2176f => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:15:30.820414 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951cfcc2-da94-4111-ba4b-888fe77f5a4c service neutron] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:15:30.831315 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951cfcc2-da94-4111-ba4b-888fe77f5a4c service neutron] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:15:30.835925 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:15:30.895389 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b79739-26cc-473e-8061-38798c68e60d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:15:30.896311 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b79739-26cc-473e-8061-38798c68e60d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:15:30.896628 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b79739-26cc-473e-8061-38798c68e60d admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:15:30.897063 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b79739-26cc-473e-8061-38798c68e60d admin admin] PATH_INFO: `/v3/users/3918887911cd4b6996cc2ac592b8f096` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:15:30.898352 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b79739-26cc-473e-8061-38798c68e60d admin admin] RBAC: Authorizing `identity:delete_user(user_id=3918887911cd4b6996cc2ac592b8f096)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:15:30.900313 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b79739-26cc-473e-8061-38798c68e60d admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:15:30.952281 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-11b79739-26cc-473e-8061-38798c68e60d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3918887911cd4b6996cc2ac592b8f096'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:15:30.959934 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-11b79739-26cc-473e-8061-38798c68e60d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3918887911cd4b6996cc2ac592b8f096'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:15:30.960811 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-11b79739-26cc-473e-8061-38798c68e60d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3918887911cd4b6996cc2ac592b8f096'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:15:30.977256 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-11b79739-26cc-473e-8061-38798c68e60d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3918887911cd4b6996cc2ac592b8f096'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:15:30.984887 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-11b79739-26cc-473e-8061-38798c68e60d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3918887911cd4b6996cc2ac592b8f096'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:15:30.991460 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1151/2316] 10.222.0.46 () {58 vars in 1223 bytes} [Thu May 26 15:15:30 2022] DELETE /identity/v3/users/3918887911cd4b6996cc2ac592b8f096 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:15:31.271636 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03389ade-b8ae-44da-98d1-6b0bd0f4d9ea admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:15:31.273129 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-03389ade-b8ae-44da-98d1-6b0bd0f4d9ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:15:31.288229 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03389ade-b8ae-44da-98d1-6b0bd0f4d9ea admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:15:31.353928 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd5d683-91d7-4c64-a9d7-2fc0b939d954 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:15:31.354798 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd5d683-91d7-4c64-a9d7-2fc0b939d954 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:15:31.355080 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd5d683-91d7-4c64-a9d7-2fc0b939d954 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:15:31.355358 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd5d683-91d7-4c64-a9d7-2fc0b939d954 admin admin] PATH_INFO: `/v3/projects/b038d8ff190a4599858bc63a0651d6d8` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:15:31.356576 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd5d683-91d7-4c64-a9d7-2fc0b939d954 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b038d8ff190a4599858bc63a0651d6d8)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:15:31.358532 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd5d683-91d7-4c64-a9d7-2fc0b939d954 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:15:31.413075 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-bcd5d683-91d7-4c64-a9d7-2fc0b939d954 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b038d8ff190a4599858bc63a0651d6d8', 'request_id': 'req-bcd5d683-91d7-4c64-a9d7-2fc0b939d954'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:15:31.424666 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-bcd5d683-91d7-4c64-a9d7-2fc0b939d954 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b038d8ff190a4599858bc63a0651d6d8', 'request_id': 'req-bcd5d683-91d7-4c64-a9d7-2fc0b939d954'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:15:31.430306 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-bcd5d683-91d7-4c64-a9d7-2fc0b939d954 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b038d8ff190a4599858bc63a0651d6d8', 'request_id': 'req-bcd5d683-91d7-4c64-a9d7-2fc0b939d954'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:15:31.441932 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-bcd5d683-91d7-4c64-a9d7-2fc0b939d954 admin admin] The token cache is being invalidate because project b038d8ff190a4599858bc63a0651d6d8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 15:15:31.442357 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-bcd5d683-91d7-4c64-a9d7-2fc0b939d954 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:15:31.442905 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-bcd5d683-91d7-4c64-a9d7-2fc0b939d954 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:15:31.444460 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1166/2317] 10.222.0.46 () {58 vars in 1232 bytes} [Thu May 26 15:15:31 2022] DELETE /identity/v3/projects/b038d8ff190a4599858bc63a0651d6d8 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:15:31.450904 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acfadf3-d071-4282-a05d-f64698261882 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:15:31.451374 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acfadf3-d071-4282-a05d-f64698261882 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:15:31.451494 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acfadf3-d071-4282-a05d-f64698261882 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:15:31.470868 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-5acfadf3-d071-4282-a05d-f64698261882 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:15:31.471861 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-5acfadf3-d071-4282-a05d-f64698261882 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:15:31.532376 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1152/2318] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:15:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:15:31.540161 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a0a596-9823-4f48-8348-2f84bcb34ad3 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:15:31.540521 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a0a596-9823-4f48-8348-2f84bcb34ad3 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:15:31.540805 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a0a596-9823-4f48-8348-2f84bcb34ad3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:15:31.560542 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-41a0a596-9823-4f48-8348-2f84bcb34ad3 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:15:31.561847 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-41a0a596-9823-4f48-8348-2f84bcb34ad3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:15:31.570011 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1167/2319] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:15:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:15:31.574286 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acfadf3-d071-4282-a05d-f64698261882 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:15:31.584922 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acfadf3-d071-4282-a05d-f64698261882 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:15:31.589766 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f27dbb-f092-495d-9018-b21448120f29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:15:31.590459 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f27dbb-f092-495d-9018-b21448120f29 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:15:31.590747 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f27dbb-f092-495d-9018-b21448120f29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:15:31.591031 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f27dbb-f092-495d-9018-b21448120f29 admin admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:15:31.591759 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f27dbb-f092-495d-9018-b21448120f29 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 15:15:31.592074 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f27dbb-f092-495d-9018-b21448120f29 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:15:31.593697 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f27dbb-f092-495d-9018-b21448120f29 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:15:31.606915 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1153/2320] 10.222.0.46 () {58 vars in 1152 bytes} [Thu May 26 15:15:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 15:15:31.610839 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a0a596-9823-4f48-8348-2f84bcb34ad3 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:15:31.620743 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a0a596-9823-4f48-8348-2f84bcb34ad3 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:15:31.626450 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c62933-d72a-4bf8-831f-b622b08b723d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:15:31.627128 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c62933-d72a-4bf8-831f-b622b08b723d admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:15:31.627423 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c62933-d72a-4bf8-831f-b622b08b723d admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:15:31.627700 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c62933-d72a-4bf8-831f-b622b08b723d admin admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:15:31.628448 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c62933-d72a-4bf8-831f-b622b08b723d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:15:31.630470 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c62933-d72a-4bf8-831f-b622b08b723d admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:15:31.660492 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1168/2321] 10.222.0.46 () {60 vars in 1152 bytes} [Thu May 26 15:15:31 2022] POST /identity/v3/projects => generated 383 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 15:15:31.664857 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f27dbb-f092-495d-9018-b21448120f29 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:15:31.675965 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f27dbb-f092-495d-9018-b21448120f29 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:15:31.687991 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:15:31.735023 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0cbd1f-fa06-4632-bae5-2dd80af7b804 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:15:31.735654 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0cbd1f-fa06-4632-bae5-2dd80af7b804 admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:15:31.735941 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0cbd1f-fa06-4632-bae5-2dd80af7b804 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:15:31.736235 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0cbd1f-fa06-4632-bae5-2dd80af7b804 admin admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:15:31.737069 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0cbd1f-fa06-4632-bae5-2dd80af7b804 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:15:31.739074 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0cbd1f-fa06-4632-bae5-2dd80af7b804 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:15:31.761487 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1154/2322] 10.222.0.46 () {60 vars in 1143 bytes} [Thu May 26 15:15:31 2022] POST /identity/v3/users => generated 292 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 15:15:31.765806 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c62933-d72a-4bf8-831f-b622b08b723d admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:15:31.776172 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c62933-d72a-4bf8-831f-b622b08b723d admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:15:31.781317 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f50344-4331-49ca-8f8e-56fae52caa09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:15:31.782080 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f50344-4331-49ca-8f8e-56fae52caa09 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:15:31.782362 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f50344-4331-49ca-8f8e-56fae52caa09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:15:31.782644 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f50344-4331-49ca-8f8e-56fae52caa09 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:15:31.783380 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f50344-4331-49ca-8f8e-56fae52caa09 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:15:31.785059 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f50344-4331-49ca-8f8e-56fae52caa09 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:15:31.797069 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1169/2323] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 15:15:31 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 15:15:31.801327 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0cbd1f-fa06-4632-bae5-2dd80af7b804 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:15:31.812941 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0cbd1f-fa06-4632-bae5-2dd80af7b804 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:15:31.817946 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c121c40c-6fc8-419c-9f09-7d72362514c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:15:31.818719 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c121c40c-6fc8-419c-9f09-7d72362514c9 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:15:31.819007 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c121c40c-6fc8-419c-9f09-7d72362514c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:15:31.819286 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c121c40c-6fc8-419c-9f09-7d72362514c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:15:31.820023 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c121c40c-6fc8-419c-9f09-7d72362514c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:15:31.821654 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c121c40c-6fc8-419c-9f09-7d72362514c9 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:15:31.831895 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1155/2324] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 15:15:31 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 15:15:31.836173 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f50344-4331-49ca-8f8e-56fae52caa09 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:15:31.848374 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f50344-4331-49ca-8f8e-56fae52caa09 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:15:31.853858 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde75524-512c-45b9-b585-bc1a40a923f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:15:31.854513 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde75524-512c-45b9-b585-bc1a40a923f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:15:31.854794 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde75524-512c-45b9-b585-bc1a40a923f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:15:31.855067 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde75524-512c-45b9-b585-bc1a40a923f9 admin admin] PATH_INFO: `/v3/projects/9dbb11e7badf4f59b8f905f0439d70e5/users/ea34d5678a6e42eea397e1360d02209d/roles/d4088d1845a24fbd9bd8975ce0b1d3d0` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:15:31.873505 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde75524-512c-45b9-b585-bc1a40a923f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9dbb11e7badf4f59b8f905f0439d70e5, user_id=ea34d5678a6e42eea397e1360d02209d, role_id=d4088d1845a24fbd9bd8975ce0b1d3d0)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:15:31.876238 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde75524-512c-45b9-b585-bc1a40a923f9 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:15:31.884875 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:DELETE=11,keystone:SELECT=85,keystone:INSERT=4 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:15:31.893742 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1170/2325] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 15:15:31 2022] PUT /identity/v3/projects/9dbb11e7badf4f59b8f905f0439d70e5/users/ea34d5678a6e42eea397e1360d02209d/roles/d4088d1845a24fbd9bd8975ce0b1d3d0 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:15:31.905237 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a56ed0-703d-4293-a2b7-bbcc366d1a27 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:15:31.905558 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a56ed0-703d-4293-a2b7-bbcc366d1a27 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:15:31.905875 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a56ed0-703d-4293-a2b7-bbcc366d1a27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:15:31.934730 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-12a56ed0-703d-4293-a2b7-bbcc366d1a27 None None] MFA Rules not processed for user `ea34d5678a6e42eea397e1360d02209d`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:15:31.935940 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-12a56ed0-703d-4293-a2b7-bbcc366d1a27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:15:32.001847 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1156/2326] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:15:31 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:15:32.006098 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde75524-512c-45b9-b585-bc1a40a923f9 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:15:32.068363 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde75524-512c-45b9-b585-bc1a40a923f9 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:15:32.073093 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299a6658-0be4-4bad-8ded-7bfc197a0af9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:15:32.073784 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299a6658-0be4-4bad-8ded-7bfc197a0af9 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:15:32.074070 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299a6658-0be4-4bad-8ded-7bfc197a0af9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:15:32.074347 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299a6658-0be4-4bad-8ded-7bfc197a0af9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:15:32.075071 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299a6658-0be4-4bad-8ded-7bfc197a0af9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:15:32.076969 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299a6658-0be4-4bad-8ded-7bfc197a0af9 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:15:32.103326 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1171/2327] 10.222.0.46 () {60 vars in 1152 bytes} [Thu May 26 15:15:32 2022] POST /identity/v3/projects => generated 381 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 15:15:32.108202 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a56ed0-703d-4293-a2b7-bbcc366d1a27 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:15:32.120775 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a56ed0-703d-4293-a2b7-bbcc366d1a27 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:15:32.182575 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d64f13-cbcc-4955-a968-65ec2435755f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:15:32.183257 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d64f13-cbcc-4955-a968-65ec2435755f admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:15:32.183539 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d64f13-cbcc-4955-a968-65ec2435755f admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:15:32.183816 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d64f13-cbcc-4955-a968-65ec2435755f admin admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:15:32.184515 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d64f13-cbcc-4955-a968-65ec2435755f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:15:32.186515 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d64f13-cbcc-4955-a968-65ec2435755f admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:15:32.207200 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1157/2328] 10.222.0.46 () {60 vars in 1143 bytes} [Thu May 26 15:15:32 2022] POST /identity/v3/users => generated 297 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 15:15:32.211465 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299a6658-0be4-4bad-8ded-7bfc197a0af9 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:15:32.224201 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299a6658-0be4-4bad-8ded-7bfc197a0af9 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:15:32.228841 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a612c8-9167-4bce-afb9-45ac164b6870 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:15:32.229533 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a612c8-9167-4bce-afb9-45ac164b6870 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:15:32.229823 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a612c8-9167-4bce-afb9-45ac164b6870 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:15:32.230102 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a612c8-9167-4bce-afb9-45ac164b6870 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:15:32.230836 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a612c8-9167-4bce-afb9-45ac164b6870 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:15:32.233057 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a612c8-9167-4bce-afb9-45ac164b6870 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:15:32.244093 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1172/2329] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 15:15:32 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 15:15:32.249001 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d64f13-cbcc-4955-a968-65ec2435755f admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:15:32.277049 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d64f13-cbcc-4955-a968-65ec2435755f admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:15:32.286629 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e632f8-0f8b-424a-a03a-3bf658540953 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:15:32.287668 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e632f8-0f8b-424a-a03a-3bf658540953 admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:15:32.287953 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e632f8-0f8b-424a-a03a-3bf658540953 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:15:32.288234 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e632f8-0f8b-424a-a03a-3bf658540953 admin admin] PATH_INFO: `/v3/projects/9c71d41efe7640f6afb0fca287e37c57/users/e4f66517cfb644d2af33fc152a7203ba/roles/a2e551773cbd4df084020676865b8222` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:15:32.301507 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:15:32.307132 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e632f8-0f8b-424a-a03a-3bf658540953 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c71d41efe7640f6afb0fca287e37c57, user_id=e4f66517cfb644d2af33fc152a7203ba, role_id=a2e551773cbd4df084020676865b8222)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:15:32.310089 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e632f8-0f8b-424a-a03a-3bf658540953 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:15:32.330245 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1158/2330] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 15:15:32 2022] PUT /identity/v3/projects/9c71d41efe7640f6afb0fca287e37c57/users/e4f66517cfb644d2af33fc152a7203ba/roles/a2e551773cbd4df084020676865b8222 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:15:32.334407 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a612c8-9167-4bce-afb9-45ac164b6870 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:15:32.346446 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a612c8-9167-4bce-afb9-45ac164b6870 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:15:32.417254 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a03fa24-2efa-4d48-b1fe-1625dd4d720b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:15:32.417977 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a03fa24-2efa-4d48-b1fe-1625dd4d720b admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:15:32.418260 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a03fa24-2efa-4d48-b1fe-1625dd4d720b admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:15:32.418584 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a03fa24-2efa-4d48-b1fe-1625dd4d720b admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:15:32.419678 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a03fa24-2efa-4d48-b1fe-1625dd4d720b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:15:32.421340 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a03fa24-2efa-4d48-b1fe-1625dd4d720b admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:15:32.432477 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1173/2331] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 15:15:32 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 15:15:32.436874 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e632f8-0f8b-424a-a03a-3bf658540953 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:15:32.447923 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e632f8-0f8b-424a-a03a-3bf658540953 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:15:32.453319 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc83899-3220-47d0-9e18-8b6e9ffcf764 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:15:32.454248 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc83899-3220-47d0-9e18-8b6e9ffcf764 admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:15:32.454528 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc83899-3220-47d0-9e18-8b6e9ffcf764 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:15:32.454805 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc83899-3220-47d0-9e18-8b6e9ffcf764 admin admin] PATH_INFO: `/v3/projects/9c71d41efe7640f6afb0fca287e37c57/users/e4f66517cfb644d2af33fc152a7203ba/roles/a2e551773cbd4df084020676865b8222` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:15:32.457383 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc83899-3220-47d0-9e18-8b6e9ffcf764 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c71d41efe7640f6afb0fca287e37c57, user_id=e4f66517cfb644d2af33fc152a7203ba, role_id=a2e551773cbd4df084020676865b8222)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:15:32.460430 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc83899-3220-47d0-9e18-8b6e9ffcf764 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:15:32.477760 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1159/2332] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 15:15:32 2022] PUT /identity/v3/projects/9c71d41efe7640f6afb0fca287e37c57/users/e4f66517cfb644d2af33fc152a7203ba/roles/a2e551773cbd4df084020676865b8222 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:15:32.488316 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dbd75b-675b-47d4-9ff9-a417756b4aea None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:15:32.493733 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dbd75b-675b-47d4-9ff9-a417756b4aea None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:15:32.494129 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dbd75b-675b-47d4-9ff9-a417756b4aea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:15:32.524523 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-34dbd75b-675b-47d4-9ff9-a417756b4aea None None] MFA Rules not processed for user `e4f66517cfb644d2af33fc152a7203ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:15:32.525775 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-34dbd75b-675b-47d4-9ff9-a417756b4aea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:15:32.604971 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:15:32.617566 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1174/2333] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:15:32 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 15:15:32.629823 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc83899-3220-47d0-9e18-8b6e9ffcf764 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:15:32.631563 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-0cc83899-3220-47d0-9e18-8b6e9ffcf764 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:15:32.696706 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc83899-3220-47d0-9e18-8b6e9ffcf764 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:15:32.700918 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19483a6-cd97-43ea-b690-71017abdf002 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5b5b35614a547d783e6e77ddb8c1c03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:15:32.701597 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19483a6-cd97-43ea-b690-71017abdf002 service cinder] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:15:32.701880 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19483a6-cd97-43ea-b690-71017abdf002 service cinder] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:15:32.702156 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19483a6-cd97-43ea-b690-71017abdf002 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:15:32.711524 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19483a6-cd97-43ea-b690-71017abdf002 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:15:32.712964 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c19483a6-cd97-43ea-b690-71017abdf002 service cinder] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:15:32.723872 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1160/2334] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:15:32 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 15:15:32.804998 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34dbd75b-675b-47d4-9ff9-a417756b4aea None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:15:32.815852 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34dbd75b-675b-47d4-9ff9-a417756b4aea None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:15:32.819451 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ccfe85-261b-44ee-a225-2ea82eb8e7da service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5b5b35614a547d783e6e77ddb8c1c03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:15:32.820103 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ccfe85-261b-44ee-a225-2ea82eb8e7da service cinder] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:15:32.820389 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ccfe85-261b-44ee-a225-2ea82eb8e7da service cinder] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:15:32.820670 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ccfe85-261b-44ee-a225-2ea82eb8e7da service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:15:32.888940 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ccfe85-261b-44ee-a225-2ea82eb8e7da service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:15:32.890796 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ccfe85-261b-44ee-a225-2ea82eb8e7da service cinder] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:15:32.901728 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1175/2335] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:15:32 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 15:15:33.196160 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19483a6-cd97-43ea-b690-71017abdf002 service cinder] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:15:33.273934 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19483a6-cd97-43ea-b690-71017abdf002 service cinder] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:15:33.278541 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39531236-3af0-43ea-a33f-ee922dc33d21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:15:33.279412 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39531236-3af0-43ea-a33f-ee922dc33d21 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:15:33.281830 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39531236-3af0-43ea-a33f-ee922dc33d21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:15:33.282163 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39531236-3af0-43ea-a33f-ee922dc33d21 admin admin] PATH_INFO: `/v3/users/ea34d5678a6e42eea397e1360d02209d` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:15:33.283580 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39531236-3af0-43ea-a33f-ee922dc33d21 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ea34d5678a6e42eea397e1360d02209d)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:15:33.285657 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39531236-3af0-43ea-a33f-ee922dc33d21 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:15:33.347052 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-39531236-3af0-43ea-a33f-ee922dc33d21 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ea34d5678a6e42eea397e1360d02209d'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:15:33.356453 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-39531236-3af0-43ea-a33f-ee922dc33d21 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ea34d5678a6e42eea397e1360d02209d'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:15:33.357142 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-39531236-3af0-43ea-a33f-ee922dc33d21 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ea34d5678a6e42eea397e1360d02209d'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:15:33.378108 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-39531236-3af0-43ea-a33f-ee922dc33d21 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ea34d5678a6e42eea397e1360d02209d'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:15:33.385823 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-39531236-3af0-43ea-a33f-ee922dc33d21 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ea34d5678a6e42eea397e1360d02209d'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:15:33.387665 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:15:33.394377 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1161/2336] 10.222.0.46 () {58 vars in 1223 bytes} [Thu May 26 15:15:33 2022] DELETE /identity/v3/users/ea34d5678a6e42eea397e1360d02209d => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 15:15:33.401658 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc4b87e-6e90-4366-bf3d-b8a41f4589d4 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:15:33.402245 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc4b87e-6e90-4366-bf3d-b8a41f4589d4 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:15:33.402245 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc4b87e-6e90-4366-bf3d-b8a41f4589d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:15:33.424450 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-2dc4b87e-6e90-4366-bf3d-b8a41f4589d4 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:15:33.425723 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-2dc4b87e-6e90-4366-bf3d-b8a41f4589d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:15:33.497068 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1176/2337] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:15:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:15:33.510400 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39531236-3af0-43ea-a33f-ee922dc33d21 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:15:33.512552 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-39531236-3af0-43ea-a33f-ee922dc33d21 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:15:33.597377 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39531236-3af0-43ea-a33f-ee922dc33d21 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:15:33.602458 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009ea7e8-412d-4a36-8f27-1d50a3e1c5a3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0156a2f6f3794f479e76b85bdbbe5c26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:15:33.603117 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009ea7e8-412d-4a36-8f27-1d50a3e1c5a3 service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:15:33.603397 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009ea7e8-412d-4a36-8f27-1d50a3e1c5a3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:15:33.603673 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009ea7e8-412d-4a36-8f27-1d50a3e1c5a3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:15:33.616272 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009ea7e8-412d-4a36-8f27-1d50a3e1c5a3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:15:33.618240 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009ea7e8-412d-4a36-8f27-1d50a3e1c5a3 service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:15:33.635980 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1162/2338] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:15:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 15:15:33.958229 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc4b87e-6e90-4366-bf3d-b8a41f4589d4 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:15:33.959769 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-2dc4b87e-6e90-4366-bf3d-b8a41f4589d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:15:33.973256 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc4b87e-6e90-4366-bf3d-b8a41f4589d4 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:15:33.978653 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73003b5-e6a6-4dce-838e-baeb6d4bc3b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:15:33.979932 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73003b5-e6a6-4dce-838e-baeb6d4bc3b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:15:33.980228 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73003b5-e6a6-4dce-838e-baeb6d4bc3b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:15:33.980505 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73003b5-e6a6-4dce-838e-baeb6d4bc3b0 admin admin] PATH_INFO: `/v3/projects/9dbb11e7badf4f59b8f905f0439d70e5` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:15:33.982686 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73003b5-e6a6-4dce-838e-baeb6d4bc3b0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9dbb11e7badf4f59b8f905f0439d70e5)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:15:33.987119 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73003b5-e6a6-4dce-838e-baeb6d4bc3b0 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:15:34.049324 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-e73003b5-e6a6-4dce-838e-baeb6d4bc3b0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9dbb11e7badf4f59b8f905f0439d70e5', 'request_id': 'req-e73003b5-e6a6-4dce-838e-baeb6d4bc3b0'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:15:34.071973 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-e73003b5-e6a6-4dce-838e-baeb6d4bc3b0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9dbb11e7badf4f59b8f905f0439d70e5', 'request_id': 'req-e73003b5-e6a6-4dce-838e-baeb6d4bc3b0'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:15:34.082174 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-e73003b5-e6a6-4dce-838e-baeb6d4bc3b0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9dbb11e7badf4f59b8f905f0439d70e5', 'request_id': 'req-e73003b5-e6a6-4dce-838e-baeb6d4bc3b0'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:15:34.096704 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-e73003b5-e6a6-4dce-838e-baeb6d4bc3b0 admin admin] The token cache is being invalidate because project 9dbb11e7badf4f59b8f905f0439d70e5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 15:15:34.096993 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-e73003b5-e6a6-4dce-838e-baeb6d4bc3b0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:15:34.097692 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-e73003b5-e6a6-4dce-838e-baeb6d4bc3b0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:15:34.099253 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1177/2339] 10.222.0.46 () {58 vars in 1232 bytes} [Thu May 26 15:15:33 2022] DELETE /identity/v3/projects/9dbb11e7badf4f59b8f905f0439d70e5 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:15:34.104111 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009ea7e8-412d-4a36-8f27-1d50a3e1c5a3 service neutron] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:15:34.123969 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009ea7e8-412d-4a36-8f27-1d50a3e1c5a3 service neutron] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:15:34.204295 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e6b120-7ec5-45ea-8635-c6a8f41364e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:15:34.205633 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e6b120-7ec5-45ea-8635-c6a8f41364e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:15:34.205906 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e6b120-7ec5-45ea-8635-c6a8f41364e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:15:34.206189 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e6b120-7ec5-45ea-8635-c6a8f41364e8 admin admin] PATH_INFO: `/v3/users/e4f66517cfb644d2af33fc152a7203ba` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:15:34.207444 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e6b120-7ec5-45ea-8635-c6a8f41364e8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e4f66517cfb644d2af33fc152a7203ba)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:15:34.209398 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e6b120-7ec5-45ea-8635-c6a8f41364e8 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:15:34.267139 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-36e6b120-7ec5-45ea-8635-c6a8f41364e8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e4f66517cfb644d2af33fc152a7203ba'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:15:34.276203 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-36e6b120-7ec5-45ea-8635-c6a8f41364e8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e4f66517cfb644d2af33fc152a7203ba'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:15:34.276841 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-36e6b120-7ec5-45ea-8635-c6a8f41364e8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e4f66517cfb644d2af33fc152a7203ba'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:15:34.291066 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-36e6b120-7ec5-45ea-8635-c6a8f41364e8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e4f66517cfb644d2af33fc152a7203ba'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:15:34.298356 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-36e6b120-7ec5-45ea-8635-c6a8f41364e8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e4f66517cfb644d2af33fc152a7203ba'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:15:34.304903 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1163/2340] 10.222.0.46 () {58 vars in 1223 bytes} [Thu May 26 15:15:34 2022] DELETE /identity/v3/users/e4f66517cfb644d2af33fc152a7203ba => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:15:34.650441 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73003b5-e6a6-4dce-838e-baeb6d4bc3b0 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:15:34.652625 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-e73003b5-e6a6-4dce-838e-baeb6d4bc3b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:15:34.667683 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73003b5-e6a6-4dce-838e-baeb6d4bc3b0 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:15:34.687026 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:15:34.744286 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fac886-d64c-4f50-bcc9-968a480618da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:15:34.745237 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fac886-d64c-4f50-bcc9-968a480618da admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:15:34.745584 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fac886-d64c-4f50-bcc9-968a480618da admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:15:34.745911 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fac886-d64c-4f50-bcc9-968a480618da admin admin] PATH_INFO: `/v3/projects/9c71d41efe7640f6afb0fca287e37c57` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:15:34.747229 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84fac886-d64c-4f50-bcc9-968a480618da admin admin] RBAC: Authorizing `identity:delete_project(project_id=9c71d41efe7640f6afb0fca287e37c57)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:15:34.749168 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84fac886-d64c-4f50-bcc9-968a480618da admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:15:34.813754 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-84fac886-d64c-4f50-bcc9-968a480618da admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9c71d41efe7640f6afb0fca287e37c57', 'request_id': 'req-84fac886-d64c-4f50-bcc9-968a480618da'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:15:34.823618 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-84fac886-d64c-4f50-bcc9-968a480618da admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9c71d41efe7640f6afb0fca287e37c57', 'request_id': 'req-84fac886-d64c-4f50-bcc9-968a480618da'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:15:34.831688 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-84fac886-d64c-4f50-bcc9-968a480618da admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9c71d41efe7640f6afb0fca287e37c57', 'request_id': 'req-84fac886-d64c-4f50-bcc9-968a480618da'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:15:34.845711 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-84fac886-d64c-4f50-bcc9-968a480618da admin admin] The token cache is being invalidate because project 9c71d41efe7640f6afb0fca287e37c57 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 15:15:34.846000 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-84fac886-d64c-4f50-bcc9-968a480618da admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:15:34.846658 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-84fac886-d64c-4f50-bcc9-968a480618da admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:15:34.848625 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1178/2341] 10.222.0.46 () {58 vars in 1232 bytes} [Thu May 26 15:15:34 2022] DELETE /identity/v3/projects/9c71d41efe7640f6afb0fca287e37c57 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:15:34.854199 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b289dd-58ae-407a-bc78-1f86b4f20b0a None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:15:34.854672 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b289dd-58ae-407a-bc78-1f86b4f20b0a None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:15:34.854853 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63b289dd-58ae-407a-bc78-1f86b4f20b0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:15:34.874555 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-63b289dd-58ae-407a-bc78-1f86b4f20b0a None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:15:34.875942 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-63b289dd-58ae-407a-bc78-1f86b4f20b0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:15:34.890926 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:15:34.951567 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1164/2342] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:15:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 15:15:34.963567 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f8d816-00fa-4d40-9025-08a695b0c4ce None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:15:34.963865 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f8d816-00fa-4d40-9025-08a695b0c4ce None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:15:34.964263 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f8d816-00fa-4d40-9025-08a695b0c4ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:15:34.989641 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-09f8d816-00fa-4d40-9025-08a695b0c4ce None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:15:34.990950 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-09f8d816-00fa-4d40-9025-08a695b0c4ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:15:34.999630 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1179/2343] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:15:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:15:35.004328 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b289dd-58ae-407a-bc78-1f86b4f20b0a None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:15:35.017288 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63b289dd-58ae-407a-bc78-1f86b4f20b0a None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:15:35.022632 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5fe652-483a-4dfb-83bc-4cfb93f47912 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:15:35.023300 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5fe652-483a-4dfb-83bc-4cfb93f47912 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:15:35.023676 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5fe652-483a-4dfb-83bc-4cfb93f47912 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:15:35.023984 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5fe652-483a-4dfb-83bc-4cfb93f47912 admin admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:15:35.024807 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5fe652-483a-4dfb-83bc-4cfb93f47912 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 15:15:35.025126 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5fe652-483a-4dfb-83bc-4cfb93f47912 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:15:35.026850 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5fe652-483a-4dfb-83bc-4cfb93f47912 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:15:35.040021 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1165/2344] 10.222.0.46 () {58 vars in 1152 bytes} [Thu May 26 15:15:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 15:15:35.044553 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f8d816-00fa-4d40-9025-08a695b0c4ce None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:15:35.054195 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f8d816-00fa-4d40-9025-08a695b0c4ce None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:15:35.059524 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb65d94-34b9-44fc-90b5-7637a86e12f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:15:35.060290 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb65d94-34b9-44fc-90b5-7637a86e12f3 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:15:35.060593 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb65d94-34b9-44fc-90b5-7637a86e12f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:15:35.060887 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb65d94-34b9-44fc-90b5-7637a86e12f3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:15:35.061610 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb65d94-34b9-44fc-90b5-7637a86e12f3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:15:35.063663 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb65d94-34b9-44fc-90b5-7637a86e12f3 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:15:35.092531 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1180/2345] 10.222.0.46 () {60 vars in 1152 bytes} [Thu May 26 15:15:35 2022] POST /identity/v3/projects => generated 367 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 15:15:35.096777 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5fe652-483a-4dfb-83bc-4cfb93f47912 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:15:35.107514 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5fe652-483a-4dfb-83bc-4cfb93f47912 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:15:35.181627 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc86a5d-b4b5-42a3-885d-5050f6f4b3a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:15:35.182437 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc86a5d-b4b5-42a3-885d-5050f6f4b3a3 admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:15:35.182747 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc86a5d-b4b5-42a3-885d-5050f6f4b3a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:15:35.183066 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc86a5d-b4b5-42a3-885d-5050f6f4b3a3 admin admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:15:35.184401 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc86a5d-b4b5-42a3-885d-5050f6f4b3a3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:15:35.186415 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc86a5d-b4b5-42a3-885d-5050f6f4b3a3 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:15:35.221322 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1166/2346] 10.222.0.46 () {60 vars in 1143 bytes} [Thu May 26 15:15:35 2022] POST /identity/v3/users => generated 284 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 15:15:35.222679 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb65d94-34b9-44fc-90b5-7637a86e12f3 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:15:35.239464 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb65d94-34b9-44fc-90b5-7637a86e12f3 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:15:35.247982 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59952dc-0470-4a7b-ad16-c8fd92bfea81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:15:35.249211 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59952dc-0470-4a7b-ad16-c8fd92bfea81 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:15:35.250056 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59952dc-0470-4a7b-ad16-c8fd92bfea81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:15:35.250956 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59952dc-0470-4a7b-ad16-c8fd92bfea81 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:15:35.251344 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59952dc-0470-4a7b-ad16-c8fd92bfea81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:15:35.254045 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59952dc-0470-4a7b-ad16-c8fd92bfea81 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:15:35.276433 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1181/2347] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 15:15:35 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 15:15:35.281430 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc86a5d-b4b5-42a3-885d-5050f6f4b3a3 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:15:35.292199 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc86a5d-b4b5-42a3-885d-5050f6f4b3a3 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:15:35.297213 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4627c6fe-75b7-4d34-982d-5fe349e8b69f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:15:35.298012 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4627c6fe-75b7-4d34-982d-5fe349e8b69f admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:15:35.298259 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4627c6fe-75b7-4d34-982d-5fe349e8b69f admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:15:35.298538 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4627c6fe-75b7-4d34-982d-5fe349e8b69f admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:15:35.299285 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4627c6fe-75b7-4d34-982d-5fe349e8b69f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:15:35.300871 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4627c6fe-75b7-4d34-982d-5fe349e8b69f admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:15:35.313724 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1167/2348] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 15:15:35 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 15:15:35.317756 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59952dc-0470-4a7b-ad16-c8fd92bfea81 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:15:35.329770 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59952dc-0470-4a7b-ad16-c8fd92bfea81 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:15:35.338994 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a1438e-db1b-4e4e-937d-7217bf5f6b58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:15:35.340076 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a1438e-db1b-4e4e-937d-7217bf5f6b58 admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:15:35.340410 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a1438e-db1b-4e4e-937d-7217bf5f6b58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:15:35.340653 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a1438e-db1b-4e4e-937d-7217bf5f6b58 admin admin] PATH_INFO: `/v3/projects/85a93d9961c2466aae0e7dbd25e34876/users/76b4411926974efab5b41d1f36df636e/roles/d4088d1845a24fbd9bd8975ce0b1d3d0` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:15:35.357298 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a1438e-db1b-4e4e-937d-7217bf5f6b58 admin admin] RBAC: Authorizing `identity:create_grant(project_id=85a93d9961c2466aae0e7dbd25e34876, user_id=76b4411926974efab5b41d1f36df636e, role_id=d4088d1845a24fbd9bd8975ce0b1d3d0)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:15:35.360137 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a1438e-db1b-4e4e-937d-7217bf5f6b58 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:15:35.377334 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1182/2349] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 15:15:35 2022] PUT /identity/v3/projects/85a93d9961c2466aae0e7dbd25e34876/users/76b4411926974efab5b41d1f36df636e/roles/d4088d1845a24fbd9bd8975ce0b1d3d0 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:15:35.389960 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e662a8-a315-40e1-9146-41e6a6e773c9 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:15:35.390253 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e662a8-a315-40e1-9146-41e6a6e773c9 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:15:35.390668 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e662a8-a315-40e1-9146-41e6a6e773c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:15:35.444977 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-c1e662a8-a315-40e1-9146-41e6a6e773c9 None None] MFA Rules not processed for user `76b4411926974efab5b41d1f36df636e`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:15:35.446841 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-c1e662a8-a315-40e1-9146-41e6a6e773c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:15:35.528539 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1168/2350] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:15:35 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:15:35.542330 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a1438e-db1b-4e4e-937d-7217bf5f6b58 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:15:35.547106 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-c3a1438e-db1b-4e4e-937d-7217bf5f6b58 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:15:35.603320 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:15:35.623782 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a1438e-db1b-4e4e-937d-7217bf5f6b58 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:15:35.629470 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8745d0ed-11f8-4564-a285-12c921ae9c6d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a436dbdee624f44ad76cca8416b0ad1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:15:35.630274 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8745d0ed-11f8-4564-a285-12c921ae9c6d service glance] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:15:35.631008 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8745d0ed-11f8-4564-a285-12c921ae9c6d service glance] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:15:35.631008 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8745d0ed-11f8-4564-a285-12c921ae9c6d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:15:35.643708 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8745d0ed-11f8-4564-a285-12c921ae9c6d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:15:35.645184 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8745d0ed-11f8-4564-a285-12c921ae9c6d service glance] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:15:35.655335 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1183/2351] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:15:35 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 15:15:45.524829 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=90,keystone:INSERT=3 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:15:45.650597 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=6 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:15:48.856971 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e662a8-a315-40e1-9146-41e6a6e773c9 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:15:48.858828 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-c1e662a8-a315-40e1-9146-41e6a6e773c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:15:48.938308 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e662a8-a315-40e1-9146-41e6a6e773c9 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:15:48.942123 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a00d8e3-578e-4adf-85d3-3a6d47e18cda service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0156a2f6f3794f479e76b85bdbbe5c26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:15:48.942785 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a00d8e3-578e-4adf-85d3-3a6d47e18cda service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:15:48.943071 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a00d8e3-578e-4adf-85d3-3a6d47e18cda service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:15:48.943359 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a00d8e3-578e-4adf-85d3-3a6d47e18cda service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:15:48.945229 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-3a00d8e3-578e-4adf-85d3-3a6d47e18cda service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:15:49.023747 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a00d8e3-578e-4adf-85d3-3a6d47e18cda service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:15:49.025313 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a00d8e3-578e-4adf-85d3-3a6d47e18cda service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:15:49.039783 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1169/2352] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:15:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 15:15:52.957025 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8745d0ed-11f8-4564-a285-12c921ae9c6d service glance] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:15:52.958576 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-8745d0ed-11f8-4564-a285-12c921ae9c6d service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:15:52.976100 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8745d0ed-11f8-4564-a285-12c921ae9c6d service glance] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:15:52.986212 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d664e6-de30-4b1e-a263-b7f0ce26fbe9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:15:52.987224 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d664e6-de30-4b1e-a263-b7f0ce26fbe9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:15:52.987514 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d664e6-de30-4b1e-a263-b7f0ce26fbe9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:15:52.987797 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d664e6-de30-4b1e-a263-b7f0ce26fbe9 admin admin] PATH_INFO: `/v3/users/e95b6425286f4235b9d1802acddac4b8` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:15:52.989067 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d664e6-de30-4b1e-a263-b7f0ce26fbe9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e95b6425286f4235b9d1802acddac4b8)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:15:52.991279 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d664e6-de30-4b1e-a263-b7f0ce26fbe9 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:15:53.048359 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-77d664e6-de30-4b1e-a263-b7f0ce26fbe9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e95b6425286f4235b9d1802acddac4b8'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:15:53.060537 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-77d664e6-de30-4b1e-a263-b7f0ce26fbe9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e95b6425286f4235b9d1802acddac4b8'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:15:53.061373 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-77d664e6-de30-4b1e-a263-b7f0ce26fbe9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e95b6425286f4235b9d1802acddac4b8'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:15:53.076119 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-77d664e6-de30-4b1e-a263-b7f0ce26fbe9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e95b6425286f4235b9d1802acddac4b8'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:15:53.083329 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-77d664e6-de30-4b1e-a263-b7f0ce26fbe9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e95b6425286f4235b9d1802acddac4b8'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:15:53.091065 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1184/2353] 10.222.0.46 () {58 vars in 1223 bytes} [Thu May 26 15:15:52 2022] DELETE /identity/v3/users/e95b6425286f4235b9d1802acddac4b8 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 15:15:53.295104 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a00d8e3-578e-4adf-85d3-3a6d47e18cda service neutron] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:15:53.315261 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a00d8e3-578e-4adf-85d3-3a6d47e18cda service neutron] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:15:53.400367 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fd244b-6ecc-4555-9874-971b148d788d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:15:53.403092 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fd244b-6ecc-4555-9874-971b148d788d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:15:53.403092 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fd244b-6ecc-4555-9874-971b148d788d admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:15:53.403092 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fd244b-6ecc-4555-9874-971b148d788d admin admin] PATH_INFO: `/v3/projects/0910873988084bc0becc79bb99febc96` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:15:53.404151 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5fd244b-6ecc-4555-9874-971b148d788d admin admin] RBAC: Authorizing `identity:delete_project(project_id=0910873988084bc0becc79bb99febc96)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:15:53.406264 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5fd244b-6ecc-4555-9874-971b148d788d admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:15:53.480714 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-c5fd244b-6ecc-4555-9874-971b148d788d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0910873988084bc0becc79bb99febc96', 'request_id': 'req-c5fd244b-6ecc-4555-9874-971b148d788d'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:15:53.483384 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:15:53.488233 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-c5fd244b-6ecc-4555-9874-971b148d788d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0910873988084bc0becc79bb99febc96', 'request_id': 'req-c5fd244b-6ecc-4555-9874-971b148d788d'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:15:53.500862 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-c5fd244b-6ecc-4555-9874-971b148d788d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0910873988084bc0becc79bb99febc96', 'request_id': 'req-c5fd244b-6ecc-4555-9874-971b148d788d'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:15:53.540191 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-c5fd244b-6ecc-4555-9874-971b148d788d admin admin] The token cache is being invalidate because project 0910873988084bc0becc79bb99febc96 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 15:15:53.540674 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-c5fd244b-6ecc-4555-9874-971b148d788d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:15:53.541831 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-c5fd244b-6ecc-4555-9874-971b148d788d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:15:53.545007 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1170/2354] 10.222.0.46 () {58 vars in 1232 bytes} [Thu May 26 15:15:53 2022] DELETE /identity/v3/projects/0910873988084bc0becc79bb99febc96 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:15:53.550555 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c4091c-ceea-408f-aad1-eb5a5311fa12 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:15:53.551046 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c4091c-ceea-408f-aad1-eb5a5311fa12 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:15:53.551666 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c4091c-ceea-408f-aad1-eb5a5311fa12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:15:53.575506 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-80c4091c-ceea-408f-aad1-eb5a5311fa12 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:15:53.577158 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-80c4091c-ceea-408f-aad1-eb5a5311fa12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:15:53.679432 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1185/2355] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:15:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:15:53.687995 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2a1ee9-e94d-4763-a4f3-bf6f2fc32afe None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:15:53.688293 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2a1ee9-e94d-4763-a4f3-bf6f2fc32afe None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:15:53.688575 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2a1ee9-e94d-4763-a4f3-bf6f2fc32afe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:15:53.710451 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-4b2a1ee9-e94d-4763-a4f3-bf6f2fc32afe None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:15:53.711707 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-4b2a1ee9-e94d-4763-a4f3-bf6f2fc32afe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:15:53.720757 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1171/2356] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:15:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:15:53.726259 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c4091c-ceea-408f-aad1-eb5a5311fa12 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:15:53.736760 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c4091c-ceea-408f-aad1-eb5a5311fa12 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:15:53.741790 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c625be80-0adf-4acb-b6ab-113a5d91df82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:15:53.742751 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c625be80-0adf-4acb-b6ab-113a5d91df82 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:15:53.743036 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c625be80-0adf-4acb-b6ab-113a5d91df82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:15:53.743316 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c625be80-0adf-4acb-b6ab-113a5d91df82 admin admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:15:53.744075 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c625be80-0adf-4acb-b6ab-113a5d91df82 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 15:15:53.744394 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c625be80-0adf-4acb-b6ab-113a5d91df82 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:15:53.746700 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c625be80-0adf-4acb-b6ab-113a5d91df82 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:15:53.762076 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1186/2357] 10.222.0.46 () {58 vars in 1152 bytes} [Thu May 26 15:15:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 15:15:53.766780 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2a1ee9-e94d-4763-a4f3-bf6f2fc32afe None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:15:53.776628 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2a1ee9-e94d-4763-a4f3-bf6f2fc32afe None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:15:53.782605 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3290c953-ee6f-4be2-bddf-b51678d83dbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:15:53.783423 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3290c953-ee6f-4be2-bddf-b51678d83dbd admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:15:53.783820 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3290c953-ee6f-4be2-bddf-b51678d83dbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:15:53.784214 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3290c953-ee6f-4be2-bddf-b51678d83dbd admin admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:15:53.785135 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3290c953-ee6f-4be2-bddf-b51678d83dbd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:15:53.787922 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3290c953-ee6f-4be2-bddf-b51678d83dbd admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:15:53.822280 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1172/2358] 10.222.0.46 () {60 vars in 1152 bytes} [Thu May 26 15:15:53 2022] POST /identity/v3/projects => generated 365 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 15:15:53.823411 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c625be80-0adf-4acb-b6ab-113a5d91df82 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:15:53.838698 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c625be80-0adf-4acb-b6ab-113a5d91df82 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:15:53.936054 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada1dca2-7066-4b6d-806d-88f2376a3156 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:15:53.936054 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada1dca2-7066-4b6d-806d-88f2376a3156 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:15:53.936054 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada1dca2-7066-4b6d-806d-88f2376a3156 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:15:53.936054 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada1dca2-7066-4b6d-806d-88f2376a3156 admin admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:15:53.937637 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada1dca2-7066-4b6d-806d-88f2376a3156 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:15:53.941876 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ada1dca2-7066-4b6d-806d-88f2376a3156 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:15:53.953145 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:15:53.978371 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1187/2359] 10.222.0.46 () {60 vars in 1143 bytes} [Thu May 26 15:15:53 2022] POST /identity/v3/users => generated 283 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 15:15:53.986052 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3290c953-ee6f-4be2-bddf-b51678d83dbd admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:15:54.003259 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3290c953-ee6f-4be2-bddf-b51678d83dbd admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:15:54.008337 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1fabfd-d28a-45d2-811e-7fb8fda9e384 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:15:54.009015 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1fabfd-d28a-45d2-811e-7fb8fda9e384 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:15:54.009305 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1fabfd-d28a-45d2-811e-7fb8fda9e384 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:15:54.009626 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1fabfd-d28a-45d2-811e-7fb8fda9e384 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:15:54.010387 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1fabfd-d28a-45d2-811e-7fb8fda9e384 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:15:54.012111 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a1fabfd-d28a-45d2-811e-7fb8fda9e384 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:15:54.023549 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1173/2360] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 15:15:53 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 15:15:54.027498 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada1dca2-7066-4b6d-806d-88f2376a3156 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:15:54.043245 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ada1dca2-7066-4b6d-806d-88f2376a3156 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:15:54.049081 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a97f41-e7f1-4d34-bb6c-e16fd7c3b7cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:15:54.049807 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a97f41-e7f1-4d34-bb6c-e16fd7c3b7cf admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:15:54.050293 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a97f41-e7f1-4d34-bb6c-e16fd7c3b7cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:15:54.050583 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a97f41-e7f1-4d34-bb6c-e16fd7c3b7cf admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:15:54.051343 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a97f41-e7f1-4d34-bb6c-e16fd7c3b7cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:15:54.053025 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a97f41-e7f1-4d34-bb6c-e16fd7c3b7cf admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:15:54.064889 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1188/2361] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 15:15:54 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 15:15:54.069338 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1fabfd-d28a-45d2-811e-7fb8fda9e384 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:15:54.081106 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1fabfd-d28a-45d2-811e-7fb8fda9e384 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:15:54.086384 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643cbecb-42ab-4fba-84dc-bdd19001f8ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:15:54.089329 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643cbecb-42ab-4fba-84dc-bdd19001f8ab admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:15:54.089329 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643cbecb-42ab-4fba-84dc-bdd19001f8ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:15:54.089329 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643cbecb-42ab-4fba-84dc-bdd19001f8ab admin admin] PATH_INFO: `/v3/projects/85e3ebf93f1a484a8b99806c20166625/users/21888a354c334a2fb5913ca804dce67c/roles/d4088d1845a24fbd9bd8975ce0b1d3d0` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:15:54.108747 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643cbecb-42ab-4fba-84dc-bdd19001f8ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=85e3ebf93f1a484a8b99806c20166625, user_id=21888a354c334a2fb5913ca804dce67c, role_id=d4088d1845a24fbd9bd8975ce0b1d3d0)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:15:54.112537 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643cbecb-42ab-4fba-84dc-bdd19001f8ab admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:15:54.133247 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1174/2362] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 15:15:54 2022] PUT /identity/v3/projects/85e3ebf93f1a484a8b99806c20166625/users/21888a354c334a2fb5913ca804dce67c/roles/d4088d1845a24fbd9bd8975ce0b1d3d0 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:15:54.145883 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9aa846-edc7-4006-9665-649013318dba None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:15:54.146096 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9aa846-edc7-4006-9665-649013318dba None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:15:54.146364 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9aa846-edc7-4006-9665-649013318dba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:15:54.191216 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-cc9aa846-edc7-4006-9665-649013318dba None None] MFA Rules not processed for user `21888a354c334a2fb5913ca804dce67c`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:15:54.192458 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-cc9aa846-edc7-4006-9665-649013318dba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:15:54.275621 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1189/2363] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:15:54 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:15:54.280832 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643cbecb-42ab-4fba-84dc-bdd19001f8ab admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:15:54.350235 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643cbecb-42ab-4fba-84dc-bdd19001f8ab admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:15:54.355480 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eab4bfa-bd1b-4d76-bcf7-4df82a17483c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:15:54.356201 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eab4bfa-bd1b-4d76-bcf7-4df82a17483c admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:15:54.356495 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eab4bfa-bd1b-4d76-bcf7-4df82a17483c admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:15:54.356813 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eab4bfa-bd1b-4d76-bcf7-4df82a17483c admin admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:15:54.357547 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eab4bfa-bd1b-4d76-bcf7-4df82a17483c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:15:54.359517 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eab4bfa-bd1b-4d76-bcf7-4df82a17483c admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:15:54.388834 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1175/2364] 10.222.0.46 () {60 vars in 1152 bytes} [Thu May 26 15:15:54 2022] POST /identity/v3/projects => generated 361 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 15:15:54.393130 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9aa846-edc7-4006-9665-649013318dba None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:15:54.404657 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9aa846-edc7-4006-9665-649013318dba None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:15:54.470745 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2c50a3-42cf-4ef6-938b-0ca6e06cb585 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:15:54.471462 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2c50a3-42cf-4ef6-938b-0ca6e06cb585 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:15:54.471750 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2c50a3-42cf-4ef6-938b-0ca6e06cb585 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:15:54.472028 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2c50a3-42cf-4ef6-938b-0ca6e06cb585 admin admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:15:54.472747 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2c50a3-42cf-4ef6-938b-0ca6e06cb585 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:15:54.474889 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2c50a3-42cf-4ef6-938b-0ca6e06cb585 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:15:54.500402 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1190/2365] 10.222.0.46 () {60 vars in 1143 bytes} [Thu May 26 15:15:54 2022] POST /identity/v3/users => generated 287 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 15:15:54.503290 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eab4bfa-bd1b-4d76-bcf7-4df82a17483c admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:15:54.515160 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eab4bfa-bd1b-4d76-bcf7-4df82a17483c admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:15:54.520219 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c24170-6a82-4d3b-877e-aa85ebeadf01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:15:54.520934 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c24170-6a82-4d3b-877e-aa85ebeadf01 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:15:54.521219 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c24170-6a82-4d3b-877e-aa85ebeadf01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:15:54.521525 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c24170-6a82-4d3b-877e-aa85ebeadf01 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:15:54.522269 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c24170-6a82-4d3b-877e-aa85ebeadf01 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:15:54.523928 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c24170-6a82-4d3b-877e-aa85ebeadf01 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:15:54.536707 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1176/2366] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 15:15:54 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 15:15:54.541134 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2c50a3-42cf-4ef6-938b-0ca6e06cb585 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:15:54.553354 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2c50a3-42cf-4ef6-938b-0ca6e06cb585 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:15:54.558451 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24424b30-3af1-4079-9eb7-1c92f84888a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:15:54.559354 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24424b30-3af1-4079-9eb7-1c92f84888a0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:15:54.559642 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24424b30-3af1-4079-9eb7-1c92f84888a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:15:54.559921 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24424b30-3af1-4079-9eb7-1c92f84888a0 admin admin] PATH_INFO: `/v3/projects/844cbac154554781936e96b45844de80/users/10f805a1a1f34a5c936f2fc54f217c8a/roles/a2e551773cbd4df084020676865b8222` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:15:54.577499 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24424b30-3af1-4079-9eb7-1c92f84888a0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=844cbac154554781936e96b45844de80, user_id=10f805a1a1f34a5c936f2fc54f217c8a, role_id=a2e551773cbd4df084020676865b8222)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:15:54.580241 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24424b30-3af1-4079-9eb7-1c92f84888a0 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:15:54.602993 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1191/2367] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 15:15:54 2022] PUT /identity/v3/projects/844cbac154554781936e96b45844de80/users/10f805a1a1f34a5c936f2fc54f217c8a/roles/a2e551773cbd4df084020676865b8222 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:15:54.607057 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c24170-6a82-4d3b-877e-aa85ebeadf01 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:15:54.618725 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c24170-6a82-4d3b-877e-aa85ebeadf01 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:15:54.693828 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:15:54.709108 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809bd74e-5d22-43ea-b6b4-e49fda7514b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:15:54.710264 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809bd74e-5d22-43ea-b6b4-e49fda7514b6 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:15:54.710755 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809bd74e-5d22-43ea-b6b4-e49fda7514b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:15:54.711243 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809bd74e-5d22-43ea-b6b4-e49fda7514b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:15:54.712536 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809bd74e-5d22-43ea-b6b4-e49fda7514b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:15:54.715478 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809bd74e-5d22-43ea-b6b4-e49fda7514b6 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:15:54.726535 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1177/2368] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 15:15:54 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 15:15:54.730959 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24424b30-3af1-4079-9eb7-1c92f84888a0 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:15:54.741585 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24424b30-3af1-4079-9eb7-1c92f84888a0 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:15:54.747250 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a467b8-6889-4a6e-b69b-f47a6d4f546a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:15:54.750936 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a467b8-6889-4a6e-b69b-f47a6d4f546a admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:15:54.750936 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a467b8-6889-4a6e-b69b-f47a6d4f546a admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:15:54.750936 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a467b8-6889-4a6e-b69b-f47a6d4f546a admin admin] PATH_INFO: `/v3/projects/844cbac154554781936e96b45844de80/users/10f805a1a1f34a5c936f2fc54f217c8a/roles/a2e551773cbd4df084020676865b8222` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:15:54.750936 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a467b8-6889-4a6e-b69b-f47a6d4f546a admin admin] RBAC: Authorizing `identity:create_grant(project_id=844cbac154554781936e96b45844de80, user_id=10f805a1a1f34a5c936f2fc54f217c8a, role_id=a2e551773cbd4df084020676865b8222)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:15:54.753597 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a467b8-6889-4a6e-b69b-f47a6d4f546a admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:15:54.766081 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1192/2369] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 15:15:54 2022] PUT /identity/v3/projects/844cbac154554781936e96b45844de80/users/10f805a1a1f34a5c936f2fc54f217c8a/roles/a2e551773cbd4df084020676865b8222 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:15:54.776208 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad9ca98-c706-4990-8322-90e5310a539a None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:15:54.776515 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad9ca98-c706-4990-8322-90e5310a539a None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:15:54.777554 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad9ca98-c706-4990-8322-90e5310a539a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:15:54.807581 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-2ad9ca98-c706-4990-8322-90e5310a539a None None] MFA Rules not processed for user `10f805a1a1f34a5c936f2fc54f217c8a`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:15:54.808808 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-2ad9ca98-c706-4990-8322-90e5310a539a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:15:54.885377 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1178/2370] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:15:54 2022] POST /identity/v3/auth/tokens => generated 3295 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:15:54.897699 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a467b8-6889-4a6e-b69b-f47a6d4f546a admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:15:54.900164 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-28a467b8-6889-4a6e-b69b-f47a6d4f546a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:15:54.928927 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:15:54.970396 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a467b8-6889-4a6e-b69b-f47a6d4f546a admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:15:54.974381 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7daec25d-f31f-41ca-a170-f6f1155cec08 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5b5b35614a547d783e6e77ddb8c1c03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:15:54.975052 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7daec25d-f31f-41ca-a170-f6f1155cec08 service cinder] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:15:54.975341 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7daec25d-f31f-41ca-a170-f6f1155cec08 service cinder] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:15:54.975937 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7daec25d-f31f-41ca-a170-f6f1155cec08 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:15:54.985698 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7daec25d-f31f-41ca-a170-f6f1155cec08 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:15:54.987195 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7daec25d-f31f-41ca-a170-f6f1155cec08 service cinder] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:15:54.998960 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1193/2371] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:15:54 2022] GET /identity/v3/auth/tokens => generated 3295 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 15:15:55.448943 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad9ca98-c706-4990-8322-90e5310a539a None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:15:55.528293 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad9ca98-c706-4990-8322-90e5310a539a None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:15:55.533405 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a19960-2b55-451a-a5b2-9a6c1a3d00e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:15:55.534339 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a19960-2b55-451a-a5b2-9a6c1a3d00e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:15:55.534620 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a19960-2b55-451a-a5b2-9a6c1a3d00e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:15:55.534897 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a19960-2b55-451a-a5b2-9a6c1a3d00e0 admin admin] PATH_INFO: `/v3/users/21888a354c334a2fb5913ca804dce67c` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:15:55.536098 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a19960-2b55-451a-a5b2-9a6c1a3d00e0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=21888a354c334a2fb5913ca804dce67c)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:15:55.538315 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a19960-2b55-451a-a5b2-9a6c1a3d00e0 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:15:55.594462 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-a8a19960-2b55-451a-a5b2-9a6c1a3d00e0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '21888a354c334a2fb5913ca804dce67c'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:15:55.604461 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-a8a19960-2b55-451a-a5b2-9a6c1a3d00e0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '21888a354c334a2fb5913ca804dce67c'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:15:55.605652 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-a8a19960-2b55-451a-a5b2-9a6c1a3d00e0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '21888a354c334a2fb5913ca804dce67c'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:15:55.623105 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-a8a19960-2b55-451a-a5b2-9a6c1a3d00e0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '21888a354c334a2fb5913ca804dce67c'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:15:55.627065 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:15:55.636666 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-a8a19960-2b55-451a-a5b2-9a6c1a3d00e0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '21888a354c334a2fb5913ca804dce67c'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:15:55.643706 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1179/2372] 10.222.0.46 () {58 vars in 1223 bytes} [Thu May 26 15:15:55 2022] DELETE /identity/v3/users/21888a354c334a2fb5913ca804dce67c => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 15:15:55.648762 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082b07d1-92be-4fe0-93c5-c6cbca4502b4 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:15:55.649221 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082b07d1-92be-4fe0-93c5-c6cbca4502b4 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:15:55.649594 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082b07d1-92be-4fe0-93c5-c6cbca4502b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:15:55.669761 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-082b07d1-92be-4fe0-93c5-c6cbca4502b4 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:15:55.671075 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-082b07d1-92be-4fe0-93c5-c6cbca4502b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:15:55.742860 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1194/2373] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:15:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:15:55.754254 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a19960-2b55-451a-a5b2-9a6c1a3d00e0 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:15:55.755719 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-a8a19960-2b55-451a-a5b2-9a6c1a3d00e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:15:55.826530 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a19960-2b55-451a-a5b2-9a6c1a3d00e0 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:15:55.830388 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50260b5-c59f-45a3-a87b-3902fe7b4f04 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0156a2f6f3794f479e76b85bdbbe5c26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:15:55.831093 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50260b5-c59f-45a3-a87b-3902fe7b4f04 service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:15:55.831428 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50260b5-c59f-45a3-a87b-3902fe7b4f04 service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:15:55.831705 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50260b5-c59f-45a3-a87b-3902fe7b4f04 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:15:55.844048 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50260b5-c59f-45a3-a87b-3902fe7b4f04 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:15:55.845547 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50260b5-c59f-45a3-a87b-3902fe7b4f04 service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:15:55.857582 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1180/2374] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:15:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 15:15:56.323157 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082b07d1-92be-4fe0-93c5-c6cbca4502b4 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:15:56.324806 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-082b07d1-92be-4fe0-93c5-c6cbca4502b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:15:56.339156 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082b07d1-92be-4fe0-93c5-c6cbca4502b4 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:15:56.349868 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b23aba-21d3-41ff-b9c2-e82b8a6fb485 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:15:56.351451 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b23aba-21d3-41ff-b9c2-e82b8a6fb485 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:15:56.351967 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b23aba-21d3-41ff-b9c2-e82b8a6fb485 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:15:56.352262 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b23aba-21d3-41ff-b9c2-e82b8a6fb485 admin admin] PATH_INFO: `/v3/projects/85e3ebf93f1a484a8b99806c20166625` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:15:56.354021 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b23aba-21d3-41ff-b9c2-e82b8a6fb485 admin admin] RBAC: Authorizing `identity:delete_project(project_id=85e3ebf93f1a484a8b99806c20166625)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:15:56.356781 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b23aba-21d3-41ff-b9c2-e82b8a6fb485 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:15:56.408438 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-53b23aba-21d3-41ff-b9c2-e82b8a6fb485 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '85e3ebf93f1a484a8b99806c20166625', 'request_id': 'req-53b23aba-21d3-41ff-b9c2-e82b8a6fb485'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:15:56.416741 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-53b23aba-21d3-41ff-b9c2-e82b8a6fb485 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '85e3ebf93f1a484a8b99806c20166625', 'request_id': 'req-53b23aba-21d3-41ff-b9c2-e82b8a6fb485'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:15:56.423905 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-53b23aba-21d3-41ff-b9c2-e82b8a6fb485 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '85e3ebf93f1a484a8b99806c20166625', 'request_id': 'req-53b23aba-21d3-41ff-b9c2-e82b8a6fb485'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:15:56.436495 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-53b23aba-21d3-41ff-b9c2-e82b8a6fb485 admin admin] The token cache is being invalidate because project 85e3ebf93f1a484a8b99806c20166625 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 15:15:56.436712 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-53b23aba-21d3-41ff-b9c2-e82b8a6fb485 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:15:56.437366 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-53b23aba-21d3-41ff-b9c2-e82b8a6fb485 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:15:56.438966 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1195/2375] 10.222.0.46 () {58 vars in 1232 bytes} [Thu May 26 15:15:56 2022] DELETE /identity/v3/projects/85e3ebf93f1a484a8b99806c20166625 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:15:56.443050 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50260b5-c59f-45a3-a87b-3902fe7b4f04 service neutron] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:15:56.453228 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50260b5-c59f-45a3-a87b-3902fe7b4f04 service neutron] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:15:56.515560 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db278b5-cc3e-4bf6-84a3-5993ab833278 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:15:56.516633 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db278b5-cc3e-4bf6-84a3-5993ab833278 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:15:56.516920 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db278b5-cc3e-4bf6-84a3-5993ab833278 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:15:56.517199 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db278b5-cc3e-4bf6-84a3-5993ab833278 admin admin] PATH_INFO: `/v3/users/10f805a1a1f34a5c936f2fc54f217c8a` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:15:56.518540 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db278b5-cc3e-4bf6-84a3-5993ab833278 admin admin] RBAC: Authorizing `identity:delete_user(user_id=10f805a1a1f34a5c936f2fc54f217c8a)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:15:56.520495 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db278b5-cc3e-4bf6-84a3-5993ab833278 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:15:56.587940 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-0db278b5-cc3e-4bf6-84a3-5993ab833278 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '10f805a1a1f34a5c936f2fc54f217c8a'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:15:56.598156 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-0db278b5-cc3e-4bf6-84a3-5993ab833278 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '10f805a1a1f34a5c936f2fc54f217c8a'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:15:56.598906 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-0db278b5-cc3e-4bf6-84a3-5993ab833278 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '10f805a1a1f34a5c936f2fc54f217c8a'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:15:56.613310 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-0db278b5-cc3e-4bf6-84a3-5993ab833278 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '10f805a1a1f34a5c936f2fc54f217c8a'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:15:56.622094 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-0db278b5-cc3e-4bf6-84a3-5993ab833278 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '10f805a1a1f34a5c936f2fc54f217c8a'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:15:56.629179 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1181/2376] 10.222.0.46 () {58 vars in 1223 bytes} [Thu May 26 15:15:56 2022] DELETE /identity/v3/users/10f805a1a1f34a5c936f2fc54f217c8a => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:15:57.011395 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b23aba-21d3-41ff-b9c2-e82b8a6fb485 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:15:57.013752 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-53b23aba-21d3-41ff-b9c2-e82b8a6fb485 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:15:57.032740 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b23aba-21d3-41ff-b9c2-e82b8a6fb485 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:15:57.096668 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:15:57.127650 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd8568f-a601-4f37-b4bd-4c7ee47b6e03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:15:57.129050 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd8568f-a601-4f37-b4bd-4c7ee47b6e03 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:15:57.129342 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd8568f-a601-4f37-b4bd-4c7ee47b6e03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:15:57.129659 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd8568f-a601-4f37-b4bd-4c7ee47b6e03 admin admin] PATH_INFO: `/v3/projects/844cbac154554781936e96b45844de80` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:15:57.132012 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd8568f-a601-4f37-b4bd-4c7ee47b6e03 admin admin] RBAC: Authorizing `identity:delete_project(project_id=844cbac154554781936e96b45844de80)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:15:57.135016 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd8568f-a601-4f37-b4bd-4c7ee47b6e03 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:15:57.240503 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-cfd8568f-a601-4f37-b4bd-4c7ee47b6e03 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '844cbac154554781936e96b45844de80', 'request_id': 'req-cfd8568f-a601-4f37-b4bd-4c7ee47b6e03'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:15:57.253769 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-cfd8568f-a601-4f37-b4bd-4c7ee47b6e03 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '844cbac154554781936e96b45844de80', 'request_id': 'req-cfd8568f-a601-4f37-b4bd-4c7ee47b6e03'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:15:57.264685 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-cfd8568f-a601-4f37-b4bd-4c7ee47b6e03 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '844cbac154554781936e96b45844de80', 'request_id': 'req-cfd8568f-a601-4f37-b4bd-4c7ee47b6e03'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:15:57.280631 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-cfd8568f-a601-4f37-b4bd-4c7ee47b6e03 admin admin] The token cache is being invalidate because project 844cbac154554781936e96b45844de80 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 15:15:57.280925 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-cfd8568f-a601-4f37-b4bd-4c7ee47b6e03 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:15:57.281636 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-cfd8568f-a601-4f37-b4bd-4c7ee47b6e03 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:15:57.283405 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1196/2377] 10.222.0.46 () {58 vars in 1232 bytes} [Thu May 26 15:15:57 2022] DELETE /identity/v3/projects/844cbac154554781936e96b45844de80 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:15:57.290545 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b933f2d-fe6f-4088-a9fb-ec0ce8db8c59 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:15:57.291046 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b933f2d-fe6f-4088-a9fb-ec0ce8db8c59 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:15:57.291533 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b933f2d-fe6f-4088-a9fb-ec0ce8db8c59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:15:57.317935 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-4b933f2d-fe6f-4088-a9fb-ec0ce8db8c59 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:15:57.319787 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-4b933f2d-fe6f-4088-a9fb-ec0ce8db8c59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:15:57.332146 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:15:57.414835 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1182/2378] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:15:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:15:57.417214 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e513a98e-1891-45a5-af37-9c5c02989403 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:15:57.417704 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e513a98e-1891-45a5-af37-9c5c02989403 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:15:57.417809 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e513a98e-1891-45a5-af37-9c5c02989403 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:15:57.439267 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-e513a98e-1891-45a5-af37-9c5c02989403 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:15:57.440537 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-e513a98e-1891-45a5-af37-9c5c02989403 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:15:57.449807 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1197/2379] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:15:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:15:57.454059 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b933f2d-fe6f-4088-a9fb-ec0ce8db8c59 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:15:57.464873 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b933f2d-fe6f-4088-a9fb-ec0ce8db8c59 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:15:57.470982 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae532d8-0daa-49df-ac28-7593524018c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:15:57.471667 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae532d8-0daa-49df-ac28-7593524018c6 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:15:57.471953 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae532d8-0daa-49df-ac28-7593524018c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:15:57.472231 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae532d8-0daa-49df-ac28-7593524018c6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:15:57.478709 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae532d8-0daa-49df-ac28-7593524018c6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 15:15:57.478709 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae532d8-0daa-49df-ac28-7593524018c6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:15:57.478709 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae532d8-0daa-49df-ac28-7593524018c6 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:15:57.488157 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1183/2380] 10.222.0.46 () {58 vars in 1152 bytes} [Thu May 26 15:15:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 15:15:57.493503 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e513a98e-1891-45a5-af37-9c5c02989403 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:15:57.504733 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e513a98e-1891-45a5-af37-9c5c02989403 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:15:57.510366 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef11b02-91a0-4ec5-80f5-5ddb5f84bfed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:15:57.510366 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef11b02-91a0-4ec5-80f5-5ddb5f84bfed admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:15:57.510726 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef11b02-91a0-4ec5-80f5-5ddb5f84bfed admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:15:57.511540 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef11b02-91a0-4ec5-80f5-5ddb5f84bfed admin admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:15:57.512433 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef11b02-91a0-4ec5-80f5-5ddb5f84bfed admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:15:57.514485 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef11b02-91a0-4ec5-80f5-5ddb5f84bfed admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:15:57.566929 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1198/2381] 10.222.0.46 () {60 vars in 1152 bytes} [Thu May 26 15:15:57 2022] POST /identity/v3/projects => generated 377 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 15:15:57.572438 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae532d8-0daa-49df-ac28-7593524018c6 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:15:57.583731 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae532d8-0daa-49df-ac28-7593524018c6 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:15:57.652212 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43737328-9938-4632-965f-e8185eec562a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:15:57.652937 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43737328-9938-4632-965f-e8185eec562a admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:15:57.653266 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43737328-9938-4632-965f-e8185eec562a admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:15:57.653584 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43737328-9938-4632-965f-e8185eec562a admin admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:15:57.654342 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43737328-9938-4632-965f-e8185eec562a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:15:57.656357 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43737328-9938-4632-965f-e8185eec562a admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:15:57.682397 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1184/2382] 10.222.0.46 () {60 vars in 1143 bytes} [Thu May 26 15:15:57 2022] POST /identity/v3/users => generated 289 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 15:15:57.687172 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef11b02-91a0-4ec5-80f5-5ddb5f84bfed admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:15:57.698373 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef11b02-91a0-4ec5-80f5-5ddb5f84bfed admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:15:57.703826 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9ae0d2-51cd-4b88-8d43-98ecd2ecc44b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:15:57.704505 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9ae0d2-51cd-4b88-8d43-98ecd2ecc44b admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:15:57.704789 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9ae0d2-51cd-4b88-8d43-98ecd2ecc44b admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:15:57.705072 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9ae0d2-51cd-4b88-8d43-98ecd2ecc44b admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:15:57.705885 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9ae0d2-51cd-4b88-8d43-98ecd2ecc44b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:15:57.707501 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9ae0d2-51cd-4b88-8d43-98ecd2ecc44b admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:15:57.719646 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1199/2383] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 15:15:57 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 15:15:57.723821 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43737328-9938-4632-965f-e8185eec562a admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:15:57.736759 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43737328-9938-4632-965f-e8185eec562a admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:15:57.741503 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e62e05b-ce8c-4009-8c4a-dc88c5ff4a53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:15:57.742434 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e62e05b-ce8c-4009-8c4a-dc88c5ff4a53 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:15:57.742727 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e62e05b-ce8c-4009-8c4a-dc88c5ff4a53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:15:57.743001 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e62e05b-ce8c-4009-8c4a-dc88c5ff4a53 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:15:57.743730 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e62e05b-ce8c-4009-8c4a-dc88c5ff4a53 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:15:57.745401 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e62e05b-ce8c-4009-8c4a-dc88c5ff4a53 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:15:57.758000 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1185/2384] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 15:15:57 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 15:15:57.762713 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9ae0d2-51cd-4b88-8d43-98ecd2ecc44b admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:15:57.773519 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9ae0d2-51cd-4b88-8d43-98ecd2ecc44b admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:15:57.780223 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5209fa-c77e-4fb7-a445-334eb211db18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:15:57.780223 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5209fa-c77e-4fb7-a445-334eb211db18 admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:15:57.780590 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5209fa-c77e-4fb7-a445-334eb211db18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:15:57.780949 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5209fa-c77e-4fb7-a445-334eb211db18 admin admin] PATH_INFO: `/v3/projects/2a12b93f082045bab1f500827b6c5181/users/874aa9a1b6cc4f98bbf2e45989792595/roles/d4088d1845a24fbd9bd8975ce0b1d3d0` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:15:57.798647 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5209fa-c77e-4fb7-a445-334eb211db18 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a12b93f082045bab1f500827b6c5181, user_id=874aa9a1b6cc4f98bbf2e45989792595, role_id=d4088d1845a24fbd9bd8975ce0b1d3d0)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:15:57.801596 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5209fa-c77e-4fb7-a445-334eb211db18 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:15:57.821235 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1200/2385] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 15:15:57 2022] PUT /identity/v3/projects/2a12b93f082045bab1f500827b6c5181/users/874aa9a1b6cc4f98bbf2e45989792595/roles/d4088d1845a24fbd9bd8975ce0b1d3d0 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:15:57.840098 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24db374-8743-4553-8cfd-dc4f35dd135c None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:15:57.840623 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24db374-8743-4553-8cfd-dc4f35dd135c None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:15:57.841121 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e24db374-8743-4553-8cfd-dc4f35dd135c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:15:57.886681 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-e24db374-8743-4553-8cfd-dc4f35dd135c None None] MFA Rules not processed for user `874aa9a1b6cc4f98bbf2e45989792595`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:15:57.887932 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-e24db374-8743-4553-8cfd-dc4f35dd135c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:15:57.967715 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1186/2386] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:15:57 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:15:57.972805 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5209fa-c77e-4fb7-a445-334eb211db18 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:15:58.046487 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5209fa-c77e-4fb7-a445-334eb211db18 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:15:58.052892 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba9e8a3-78bf-4c9a-8999-fd6495d61128 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:15:58.053773 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba9e8a3-78bf-4c9a-8999-fd6495d61128 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:15:58.054171 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba9e8a3-78bf-4c9a-8999-fd6495d61128 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:15:58.054699 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba9e8a3-78bf-4c9a-8999-fd6495d61128 admin admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:15:58.055563 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba9e8a3-78bf-4c9a-8999-fd6495d61128 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:15:58.057586 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba9e8a3-78bf-4c9a-8999-fd6495d61128 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:15:58.083220 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:15:58.091774 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1201/2387] 10.222.0.46 () {60 vars in 1152 bytes} [Thu May 26 15:15:57 2022] POST /identity/v3/projects => generated 375 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 15:15:58.096474 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24db374-8743-4553-8cfd-dc4f35dd135c None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:15:58.108663 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e24db374-8743-4553-8cfd-dc4f35dd135c None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:15:58.114584 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:15:58.175496 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e45662-f130-41f2-9eb5-77bd1b7da94f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:15:58.176185 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e45662-f130-41f2-9eb5-77bd1b7da94f admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:15:58.176468 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e45662-f130-41f2-9eb5-77bd1b7da94f admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:15:58.176745 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e45662-f130-41f2-9eb5-77bd1b7da94f admin admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:15:58.177490 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e45662-f130-41f2-9eb5-77bd1b7da94f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:15:58.179483 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e45662-f130-41f2-9eb5-77bd1b7da94f admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:15:58.204399 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1187/2388] 10.222.0.46 () {60 vars in 1143 bytes} [Thu May 26 15:15:58 2022] POST /identity/v3/users => generated 294 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 15:15:58.210184 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba9e8a3-78bf-4c9a-8999-fd6495d61128 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:15:58.221393 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba9e8a3-78bf-4c9a-8999-fd6495d61128 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:15:58.227477 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfda004b-da3a-41c3-9008-aa11c6a598dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:15:58.228165 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfda004b-da3a-41c3-9008-aa11c6a598dc admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:15:58.228453 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfda004b-da3a-41c3-9008-aa11c6a598dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:15:58.228737 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfda004b-da3a-41c3-9008-aa11c6a598dc admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:15:58.229504 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfda004b-da3a-41c3-9008-aa11c6a598dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:15:58.231138 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfda004b-da3a-41c3-9008-aa11c6a598dc admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:15:58.244690 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1202/2389] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 15:15:58 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 15:15:58.249780 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e45662-f130-41f2-9eb5-77bd1b7da94f admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:15:58.262010 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e45662-f130-41f2-9eb5-77bd1b7da94f admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:15:58.267120 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e0f722-965d-444f-88d6-d25aae451a0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:15:58.268031 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e0f722-965d-444f-88d6-d25aae451a0b admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:15:58.268314 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e0f722-965d-444f-88d6-d25aae451a0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:15:58.268590 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e0f722-965d-444f-88d6-d25aae451a0b admin admin] PATH_INFO: `/v3/projects/5136e98e54a74f46909ee4c9a8d98c42/users/4aba3d8abdfb427b9027067b69c0ac42/roles/a2e551773cbd4df084020676865b8222` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:15:58.292669 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e0f722-965d-444f-88d6-d25aae451a0b admin admin] RBAC: Authorizing `identity:create_grant(project_id=5136e98e54a74f46909ee4c9a8d98c42, user_id=4aba3d8abdfb427b9027067b69c0ac42, role_id=a2e551773cbd4df084020676865b8222)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:15:58.297520 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e0f722-965d-444f-88d6-d25aae451a0b admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:15:58.315319 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1188/2390] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 15:15:58 2022] PUT /identity/v3/projects/5136e98e54a74f46909ee4c9a8d98c42/users/4aba3d8abdfb427b9027067b69c0ac42/roles/a2e551773cbd4df084020676865b8222 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:15:58.321106 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfda004b-da3a-41c3-9008-aa11c6a598dc admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:15:58.333420 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfda004b-da3a-41c3-9008-aa11c6a598dc admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:15:58.405731 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4eb629-2c60-4c59-bd3f-3f719de5ba5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:15:58.406460 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4eb629-2c60-4c59-bd3f-3f719de5ba5f admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:15:58.406754 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4eb629-2c60-4c59-bd3f-3f719de5ba5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:15:58.407126 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4eb629-2c60-4c59-bd3f-3f719de5ba5f admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:15:58.407961 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4eb629-2c60-4c59-bd3f-3f719de5ba5f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:15:58.409785 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4eb629-2c60-4c59-bd3f-3f719de5ba5f admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:15:58.421953 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1203/2391] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 15:15:58 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 15:15:58.427694 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e0f722-965d-444f-88d6-d25aae451a0b admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:15:58.442628 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e0f722-965d-444f-88d6-d25aae451a0b admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:15:58.450996 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f2be74-ab55-41d2-8107-951c47292b6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:15:58.452028 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f2be74-ab55-41d2-8107-951c47292b6f admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:15:58.452344 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f2be74-ab55-41d2-8107-951c47292b6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:15:58.452593 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f2be74-ab55-41d2-8107-951c47292b6f admin admin] PATH_INFO: `/v3/projects/5136e98e54a74f46909ee4c9a8d98c42/users/4aba3d8abdfb427b9027067b69c0ac42/roles/a2e551773cbd4df084020676865b8222` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:15:58.455432 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f2be74-ab55-41d2-8107-951c47292b6f admin admin] RBAC: Authorizing `identity:create_grant(project_id=5136e98e54a74f46909ee4c9a8d98c42, user_id=4aba3d8abdfb427b9027067b69c0ac42, role_id=a2e551773cbd4df084020676865b8222)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:15:58.457853 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9f2be74-ab55-41d2-8107-951c47292b6f admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:15:58.471678 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1189/2392] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 15:15:58 2022] PUT /identity/v3/projects/5136e98e54a74f46909ee4c9a8d98c42/users/4aba3d8abdfb427b9027067b69c0ac42/roles/a2e551773cbd4df084020676865b8222 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:15:58.485274 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d3e295-f4ab-471b-a4c8-643b221eef4d None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:15:58.485718 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d3e295-f4ab-471b-a4c8-643b221eef4d None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:15:58.486043 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d3e295-f4ab-471b-a4c8-643b221eef4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:15:58.527627 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-d4d3e295-f4ab-471b-a4c8-643b221eef4d None None] MFA Rules not processed for user `4aba3d8abdfb427b9027067b69c0ac42`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:15:58.529003 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-d4d3e295-f4ab-471b-a4c8-643b221eef4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:15:58.583187 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f2be74-ab55-41d2-8107-951c47292b6f admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:15:58.584933 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-a9f2be74-ab55-41d2-8107-951c47292b6f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:15:58.640586 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1204/2393] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:15:58 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:15:58.660537 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d3e295-f4ab-471b-a4c8-643b221eef4d None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:15:58.663816 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-d4d3e295-f4ab-471b-a4c8-643b221eef4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:15:58.686167 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f2be74-ab55-41d2-8107-951c47292b6f admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:15:58.690987 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a423dc96-4386-49ef-9806-2ff2216f7dd3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0156a2f6f3794f479e76b85bdbbe5c26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:15:58.691659 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a423dc96-4386-49ef-9806-2ff2216f7dd3 service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:15:58.691950 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a423dc96-4386-49ef-9806-2ff2216f7dd3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:15:58.692234 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a423dc96-4386-49ef-9806-2ff2216f7dd3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:15:58.694996 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-a423dc96-4386-49ef-9806-2ff2216f7dd3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:15:58.738965 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d3e295-f4ab-471b-a4c8-643b221eef4d None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:15:58.744773 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77774c28-cc91-40bb-a811-cfdd32b7bb84 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5b5b35614a547d783e6e77ddb8c1c03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:15:58.746279 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77774c28-cc91-40bb-a811-cfdd32b7bb84 service cinder] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:15:58.746279 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77774c28-cc91-40bb-a811-cfdd32b7bb84 service cinder] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:15:58.746663 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77774c28-cc91-40bb-a811-cfdd32b7bb84 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:15:58.782627 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:15:58.797042 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a423dc96-4386-49ef-9806-2ff2216f7dd3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:15:58.799226 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a423dc96-4386-49ef-9806-2ff2216f7dd3 service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:15:58.811316 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1190/2394] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:15:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 15:15:58.843949 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77774c28-cc91-40bb-a811-cfdd32b7bb84 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:15:58.847142 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77774c28-cc91-40bb-a811-cfdd32b7bb84 service cinder] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:15:58.861822 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1205/2395] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:15:58 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 202 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 15:16:03.210468 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a423dc96-4386-49ef-9806-2ff2216f7dd3 service neutron] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:16:03.211981 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-a423dc96-4386-49ef-9806-2ff2216f7dd3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:16:03.228377 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a423dc96-4386-49ef-9806-2ff2216f7dd3 service neutron] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:16:03.234338 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973f9359-72b8-477a-8666-71e462b99cf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:16:03.235239 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973f9359-72b8-477a-8666-71e462b99cf1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:16:03.235518 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973f9359-72b8-477a-8666-71e462b99cf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:16:03.235793 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973f9359-72b8-477a-8666-71e462b99cf1 admin admin] PATH_INFO: `/v3/users/2ce5b5d5a12743bc80a4b191c24ff52f` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:16:03.237044 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973f9359-72b8-477a-8666-71e462b99cf1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2ce5b5d5a12743bc80a4b191c24ff52f)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:16:03.239085 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973f9359-72b8-477a-8666-71e462b99cf1 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:16:03.245617 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:16:03.297118 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-973f9359-72b8-477a-8666-71e462b99cf1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2ce5b5d5a12743bc80a4b191c24ff52f'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:16:03.306232 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-973f9359-72b8-477a-8666-71e462b99cf1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2ce5b5d5a12743bc80a4b191c24ff52f'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:16:03.307040 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-973f9359-72b8-477a-8666-71e462b99cf1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2ce5b5d5a12743bc80a4b191c24ff52f'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:16:03.323002 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-973f9359-72b8-477a-8666-71e462b99cf1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2ce5b5d5a12743bc80a4b191c24ff52f'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:16:03.332748 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-973f9359-72b8-477a-8666-71e462b99cf1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2ce5b5d5a12743bc80a4b191c24ff52f'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:16:03.340100 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1191/2396] 10.222.0.46 () {58 vars in 1223 bytes} [Thu May 26 15:16:03 2022] DELETE /identity/v3/users/2ce5b5d5a12743bc80a4b191c24ff52f => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:16:03.536684 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77774c28-cc91-40bb-a811-cfdd32b7bb84 service cinder] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:16:03.558660 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77774c28-cc91-40bb-a811-cfdd32b7bb84 service cinder] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:16:03.666658 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c547af4e-2935-41a5-aa02-51ee0d7981fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:16:03.668122 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c547af4e-2935-41a5-aa02-51ee0d7981fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:16:03.668399 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c547af4e-2935-41a5-aa02-51ee0d7981fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:16:03.668664 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c547af4e-2935-41a5-aa02-51ee0d7981fa admin admin] PATH_INFO: `/v3/projects/9780d0cae62c4f5088060b530571ebb3` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:16:03.669970 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c547af4e-2935-41a5-aa02-51ee0d7981fa admin admin] RBAC: Authorizing `identity:delete_project(project_id=9780d0cae62c4f5088060b530571ebb3)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:16:03.671922 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c547af4e-2935-41a5-aa02-51ee0d7981fa admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:16:03.729066 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-c547af4e-2935-41a5-aa02-51ee0d7981fa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9780d0cae62c4f5088060b530571ebb3', 'request_id': 'req-c547af4e-2935-41a5-aa02-51ee0d7981fa'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:16:03.738053 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-c547af4e-2935-41a5-aa02-51ee0d7981fa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9780d0cae62c4f5088060b530571ebb3', 'request_id': 'req-c547af4e-2935-41a5-aa02-51ee0d7981fa'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:16:03.745716 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-c547af4e-2935-41a5-aa02-51ee0d7981fa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9780d0cae62c4f5088060b530571ebb3', 'request_id': 'req-c547af4e-2935-41a5-aa02-51ee0d7981fa'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:16:03.758502 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-c547af4e-2935-41a5-aa02-51ee0d7981fa admin admin] The token cache is being invalidate because project 9780d0cae62c4f5088060b530571ebb3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 15:16:03.758886 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-c547af4e-2935-41a5-aa02-51ee0d7981fa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:16:03.759753 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-c547af4e-2935-41a5-aa02-51ee0d7981fa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:16:03.761728 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1206/2397] 10.222.0.46 () {58 vars in 1232 bytes} [Thu May 26 15:16:03 2022] DELETE /identity/v3/projects/9780d0cae62c4f5088060b530571ebb3 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:16:03.768486 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0893610b-4224-4fc9-8e57-5e515ee5ca90 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:16:03.769685 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0893610b-4224-4fc9-8e57-5e515ee5ca90 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:16:03.769685 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0893610b-4224-4fc9-8e57-5e515ee5ca90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:16:03.799474 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-0893610b-4224-4fc9-8e57-5e515ee5ca90 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:16:03.801500 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-0893610b-4224-4fc9-8e57-5e515ee5ca90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:16:03.804224 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c547af4e-2935-41a5-aa02-51ee0d7981fa admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:16:03.808104 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-c547af4e-2935-41a5-aa02-51ee0d7981fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:16:03.901797 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c547af4e-2935-41a5-aa02-51ee0d7981fa admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:16:03.911659 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1192/2398] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:16:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:16:03.912917 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6eefe11-30fa-4864-9c06-d84056d5765b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5b5b35614a547d783e6e77ddb8c1c03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:16:03.913647 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6eefe11-30fa-4864-9c06-d84056d5765b service cinder] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:16:03.914139 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6eefe11-30fa-4864-9c06-d84056d5765b service cinder] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:16:03.914445 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6eefe11-30fa-4864-9c06-d84056d5765b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:16:03.916537 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-c6eefe11-30fa-4864-9c06-d84056d5765b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:16:03.923921 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83ea4da-fb1f-4c32-8b8d-9e42715ccf2d None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:16:03.924368 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83ea4da-fb1f-4c32-8b8d-9e42715ccf2d None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:16:03.924848 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:16:03.925806 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83ea4da-fb1f-4c32-8b8d-9e42715ccf2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:16:03.956632 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-a83ea4da-fb1f-4c32-8b8d-9e42715ccf2d None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:16:03.958342 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-a83ea4da-fb1f-4c32-8b8d-9e42715ccf2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:16:03.973869 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1193/2399] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:16:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 15:16:03.975828 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83ea4da-fb1f-4c32-8b8d-9e42715ccf2d None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:16:03.994720 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83ea4da-fb1f-4c32-8b8d-9e42715ccf2d None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:16:04.006367 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48762c9e-4ce5-4347-9816-f5df0cb84f99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:16:04.007127 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48762c9e-4ce5-4347-9816-f5df0cb84f99 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:16:04.007416 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48762c9e-4ce5-4347-9816-f5df0cb84f99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:16:04.007695 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48762c9e-4ce5-4347-9816-f5df0cb84f99 admin admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:16:04.008595 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48762c9e-4ce5-4347-9816-f5df0cb84f99 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 15:16:04.008977 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48762c9e-4ce5-4347-9816-f5df0cb84f99 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:16:04.010450 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6eefe11-30fa-4864-9c06-d84056d5765b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:16:04.010711 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48762c9e-4ce5-4347-9816-f5df0cb84f99 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:16:04.013085 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6eefe11-30fa-4864-9c06-d84056d5765b service cinder] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:16:04.029480 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1194/2400] 10.222.0.46 () {58 vars in 1152 bytes} [Thu May 26 15:16:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 15:16:04.032758 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1207/2401] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:16:03 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 15:16:04.035341 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48762c9e-4ce5-4347-9816-f5df0cb84f99 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:16:04.050025 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48762c9e-4ce5-4347-9816-f5df0cb84f99 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:16:04.058817 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7d1566-952c-42ba-aa09-cef287e8dc3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:16:04.060102 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7d1566-952c-42ba-aa09-cef287e8dc3e admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:16:04.060579 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7d1566-952c-42ba-aa09-cef287e8dc3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:16:04.061053 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7d1566-952c-42ba-aa09-cef287e8dc3e admin admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:16:04.062341 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7d1566-952c-42ba-aa09-cef287e8dc3e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:16:04.066286 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e7d1566-952c-42ba-aa09-cef287e8dc3e admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:16:04.121208 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1195/2402] 10.222.0.46 () {60 vars in 1152 bytes} [Thu May 26 15:16:04 2022] POST /identity/v3/projects => generated 373 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 15:16:04.126389 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6eefe11-30fa-4864-9c06-d84056d5765b service cinder] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:16:04.140458 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6eefe11-30fa-4864-9c06-d84056d5765b service cinder] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:16:04.271731 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b5b559-05f4-4026-a9f6-fe4905711bf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:16:04.272455 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b5b559-05f4-4026-a9f6-fe4905711bf2 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:16:04.272820 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b5b559-05f4-4026-a9f6-fe4905711bf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:16:04.273008 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b5b559-05f4-4026-a9f6-fe4905711bf2 admin admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:16:04.274227 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b5b559-05f4-4026-a9f6-fe4905711bf2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:16:04.276824 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b5b559-05f4-4026-a9f6-fe4905711bf2 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:16:04.305776 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1208/2403] 10.222.0.46 () {60 vars in 1143 bytes} [Thu May 26 15:16:04 2022] POST /identity/v3/users => generated 287 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 15:16:04.312113 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7d1566-952c-42ba-aa09-cef287e8dc3e admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:16:04.333535 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7d1566-952c-42ba-aa09-cef287e8dc3e admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:16:04.344641 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbeccd6-06b2-4aa5-b1d8-a6924cb66b78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:16:04.346215 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbeccd6-06b2-4aa5-b1d8-a6924cb66b78 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:16:04.347004 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbeccd6-06b2-4aa5-b1d8-a6924cb66b78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:16:04.347669 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbeccd6-06b2-4aa5-b1d8-a6924cb66b78 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:16:04.349401 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbeccd6-06b2-4aa5-b1d8-a6924cb66b78 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:16:04.351968 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbeccd6-06b2-4aa5-b1d8-a6924cb66b78 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:16:04.373684 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1196/2404] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 15:16:04 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 15:16:04.377769 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b5b559-05f4-4026-a9f6-fe4905711bf2 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:16:04.396378 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b5b559-05f4-4026-a9f6-fe4905711bf2 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:16:04.405080 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb258b3a-fbd5-48b2-898f-b8699f365452 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:16:04.405080 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb258b3a-fbd5-48b2-898f-b8699f365452 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:16:04.405080 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb258b3a-fbd5-48b2-898f-b8699f365452 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:16:04.405080 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb258b3a-fbd5-48b2-898f-b8699f365452 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:16:04.405973 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb258b3a-fbd5-48b2-898f-b8699f365452 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:16:04.407085 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb258b3a-fbd5-48b2-898f-b8699f365452 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:16:04.419927 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1209/2405] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 15:16:04 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 15:16:04.424887 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbeccd6-06b2-4aa5-b1d8-a6924cb66b78 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:16:04.444565 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbeccd6-06b2-4aa5-b1d8-a6924cb66b78 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:16:04.455978 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a1b556-7bd7-4ca7-a4b0-3ffab6c4a042 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:16:04.455978 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a1b556-7bd7-4ca7-a4b0-3ffab6c4a042 admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:16:04.456326 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a1b556-7bd7-4ca7-a4b0-3ffab6c4a042 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:16:04.456805 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a1b556-7bd7-4ca7-a4b0-3ffab6c4a042 admin admin] PATH_INFO: `/v3/projects/d22a20b14bd54530b45ded9388b93724/users/a15d845b892b49efb41a6f72fe1db9f1/roles/d4088d1845a24fbd9bd8975ce0b1d3d0` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:16:04.487579 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a1b556-7bd7-4ca7-a4b0-3ffab6c4a042 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d22a20b14bd54530b45ded9388b93724, user_id=a15d845b892b49efb41a6f72fe1db9f1, role_id=d4088d1845a24fbd9bd8975ce0b1d3d0)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:16:04.492882 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a1b556-7bd7-4ca7-a4b0-3ffab6c4a042 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:16:04.512826 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1197/2406] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 15:16:04 2022] PUT /identity/v3/projects/d22a20b14bd54530b45ded9388b93724/users/a15d845b892b49efb41a6f72fe1db9f1/roles/d4088d1845a24fbd9bd8975ce0b1d3d0 => generated 0 bytes in 89 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:16:04.523623 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38eb45f-1833-42bd-a15b-9c92c1e6516d None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:16:04.523913 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38eb45f-1833-42bd-a15b-9c92c1e6516d None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:16:04.524176 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38eb45f-1833-42bd-a15b-9c92c1e6516d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:16:04.565827 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-d38eb45f-1833-42bd-a15b-9c92c1e6516d None None] MFA Rules not processed for user `a15d845b892b49efb41a6f72fe1db9f1`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:16:04.567089 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-d38eb45f-1833-42bd-a15b-9c92c1e6516d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:16:04.645273 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1210/2407] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:16:04 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:16:04.657461 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a1b556-7bd7-4ca7-a4b0-3ffab6c4a042 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:16:04.659367 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-11a1b556-7bd7-4ca7-a4b0-3ffab6c4a042 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:16:04.670703 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:16:04.755960 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a1b556-7bd7-4ca7-a4b0-3ffab6c4a042 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:16:04.762769 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed7edf33-b791-4a63-855d-1bd281833cfc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6639353c70449d1b3d10cd6487c3b00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:16:04.763452 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7edf33-b791-4a63-855d-1bd281833cfc service nova] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:16:04.763746 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7edf33-b791-4a63-855d-1bd281833cfc service nova] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:16:04.764036 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed7edf33-b791-4a63-855d-1bd281833cfc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:16:04.772896 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7edf33-b791-4a63-855d-1bd281833cfc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:16:04.774661 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed7edf33-b791-4a63-855d-1bd281833cfc service nova] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:16:04.789700 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1198/2408] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:16:04 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 15:16:11.940748 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38eb45f-1833-42bd-a15b-9c92c1e6516d None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:16:11.954426 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:16:12.026932 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38eb45f-1833-42bd-a15b-9c92c1e6516d None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:16:12.033093 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e433cde5-9fde-4e67-8263-a88d98091bfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:16:12.034076 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e433cde5-9fde-4e67-8263-a88d98091bfb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:16:12.034487 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e433cde5-9fde-4e67-8263-a88d98091bfb admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:16:12.035041 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e433cde5-9fde-4e67-8263-a88d98091bfb admin admin] PATH_INFO: `/v3/users/a15d845b892b49efb41a6f72fe1db9f1` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:16:12.036319 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e433cde5-9fde-4e67-8263-a88d98091bfb admin admin] RBAC: Authorizing `identity:delete_user(user_id=a15d845b892b49efb41a6f72fe1db9f1)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:16:12.038457 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e433cde5-9fde-4e67-8263-a88d98091bfb admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:16:12.107553 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-e433cde5-9fde-4e67-8263-a88d98091bfb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a15d845b892b49efb41a6f72fe1db9f1'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:16:12.121306 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-e433cde5-9fde-4e67-8263-a88d98091bfb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a15d845b892b49efb41a6f72fe1db9f1'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:16:12.121968 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-e433cde5-9fde-4e67-8263-a88d98091bfb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a15d845b892b49efb41a6f72fe1db9f1'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:16:12.142732 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-e433cde5-9fde-4e67-8263-a88d98091bfb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a15d845b892b49efb41a6f72fe1db9f1'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:16:12.151833 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-e433cde5-9fde-4e67-8263-a88d98091bfb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a15d845b892b49efb41a6f72fe1db9f1'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:16:12.160074 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1211/2409] 10.222.0.46 () {58 vars in 1223 bytes} [Thu May 26 15:16:11 2022] DELETE /identity/v3/users/a15d845b892b49efb41a6f72fe1db9f1 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:16:12.164725 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1106077f-6547-47fa-8ce2-fece958b242b None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:16:12.165124 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1106077f-6547-47fa-8ce2-fece958b242b None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:16:12.165226 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1106077f-6547-47fa-8ce2-fece958b242b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:16:12.184598 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-1106077f-6547-47fa-8ce2-fece958b242b None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:16:12.186310 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-1106077f-6547-47fa-8ce2-fece958b242b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:16:12.275079 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1199/2410] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:16:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:16:12.285996 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e433cde5-9fde-4e67-8263-a88d98091bfb admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:16:12.287637 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-e433cde5-9fde-4e67-8263-a88d98091bfb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:16:12.358199 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e433cde5-9fde-4e67-8263-a88d98091bfb admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:16:12.362154 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15594f71-ee16-4bde-ae9e-d1c956ab553b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0156a2f6f3794f479e76b85bdbbe5c26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:16:12.362869 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15594f71-ee16-4bde-ae9e-d1c956ab553b service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:16:12.363188 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15594f71-ee16-4bde-ae9e-d1c956ab553b service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:16:12.363352 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15594f71-ee16-4bde-ae9e-d1c956ab553b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:16:12.375208 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15594f71-ee16-4bde-ae9e-d1c956ab553b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:16:12.376656 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15594f71-ee16-4bde-ae9e-d1c956ab553b service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:16:12.387594 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1212/2411] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:16:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 15:16:12.714173 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1106077f-6547-47fa-8ce2-fece958b242b None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:16:12.716197 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-1106077f-6547-47fa-8ce2-fece958b242b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:16:12.740660 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1106077f-6547-47fa-8ce2-fece958b242b None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:16:12.747358 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38840b9e-06e3-4cf9-bd8c-d100f37b3673 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:16:12.748568 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38840b9e-06e3-4cf9-bd8c-d100f37b3673 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:16:12.748880 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38840b9e-06e3-4cf9-bd8c-d100f37b3673 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:16:12.749160 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38840b9e-06e3-4cf9-bd8c-d100f37b3673 admin admin] PATH_INFO: `/v3/projects/d22a20b14bd54530b45ded9388b93724` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:16:12.751265 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38840b9e-06e3-4cf9-bd8c-d100f37b3673 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d22a20b14bd54530b45ded9388b93724)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:16:12.753294 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38840b9e-06e3-4cf9-bd8c-d100f37b3673 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:16:12.821242 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-38840b9e-06e3-4cf9-bd8c-d100f37b3673 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd22a20b14bd54530b45ded9388b93724', 'request_id': 'req-38840b9e-06e3-4cf9-bd8c-d100f37b3673'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:16:12.829421 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-38840b9e-06e3-4cf9-bd8c-d100f37b3673 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd22a20b14bd54530b45ded9388b93724', 'request_id': 'req-38840b9e-06e3-4cf9-bd8c-d100f37b3673'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:16:12.836411 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-38840b9e-06e3-4cf9-bd8c-d100f37b3673 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd22a20b14bd54530b45ded9388b93724', 'request_id': 'req-38840b9e-06e3-4cf9-bd8c-d100f37b3673'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:16:12.855811 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-38840b9e-06e3-4cf9-bd8c-d100f37b3673 admin admin] The token cache is being invalidate because project d22a20b14bd54530b45ded9388b93724 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 15:16:12.856111 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-38840b9e-06e3-4cf9-bd8c-d100f37b3673 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:16:12.857697 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-38840b9e-06e3-4cf9-bd8c-d100f37b3673 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:16:12.859612 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1200/2412] 10.222.0.46 () {58 vars in 1232 bytes} [Thu May 26 15:16:12 2022] DELETE /identity/v3/projects/d22a20b14bd54530b45ded9388b93724 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:16:12.866076 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd69dd9e-0eae-4a37-b2aa-7229a7a2e53b None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:16:12.866363 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd69dd9e-0eae-4a37-b2aa-7229a7a2e53b None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:16:12.866715 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd69dd9e-0eae-4a37-b2aa-7229a7a2e53b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:16:12.900391 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-cd69dd9e-0eae-4a37-b2aa-7229a7a2e53b None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:16:12.904276 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-cd69dd9e-0eae-4a37-b2aa-7229a7a2e53b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:16:12.952389 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:16:12.998582 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1213/2413] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:16:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:16:13.011491 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186ae71d-2327-4e66-a38b-b98cdc8d56c4 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:16:13.011791 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186ae71d-2327-4e66-a38b-b98cdc8d56c4 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:16:13.012071 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186ae71d-2327-4e66-a38b-b98cdc8d56c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:16:13.037035 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-186ae71d-2327-4e66-a38b-b98cdc8d56c4 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:16:13.039324 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-186ae71d-2327-4e66-a38b-b98cdc8d56c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:16:13.057697 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1201/2414] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:16:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:16:13.058501 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd69dd9e-0eae-4a37-b2aa-7229a7a2e53b None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:16:13.068426 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd69dd9e-0eae-4a37-b2aa-7229a7a2e53b None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:16:13.075806 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fab9ac-fe59-4f22-b71a-f7ee361b424a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:16:13.076754 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fab9ac-fe59-4f22-b71a-f7ee361b424a admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:16:13.077317 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fab9ac-fe59-4f22-b71a-f7ee361b424a admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:16:13.077630 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fab9ac-fe59-4f22-b71a-f7ee361b424a admin admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:16:13.078372 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fab9ac-fe59-4f22-b71a-f7ee361b424a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 15:16:13.078854 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fab9ac-fe59-4f22-b71a-f7ee361b424a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:16:13.080636 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fab9ac-fe59-4f22-b71a-f7ee361b424a admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:16:13.096954 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1214/2415] 10.222.0.46 () {58 vars in 1152 bytes} [Thu May 26 15:16:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 15:16:13.101741 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186ae71d-2327-4e66-a38b-b98cdc8d56c4 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:16:13.113098 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186ae71d-2327-4e66-a38b-b98cdc8d56c4 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:16:13.119258 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5ed7f7-2073-4bd2-99d6-ed95510c21b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:16:13.120475 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5ed7f7-2073-4bd2-99d6-ed95510c21b9 admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:16:13.120759 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5ed7f7-2073-4bd2-99d6-ed95510c21b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:16:13.121036 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5ed7f7-2073-4bd2-99d6-ed95510c21b9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:16:13.121769 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5ed7f7-2073-4bd2-99d6-ed95510c21b9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:16:13.123967 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5ed7f7-2073-4bd2-99d6-ed95510c21b9 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:16:13.134937 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:16:13.167817 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1202/2416] 10.222.0.46 () {60 vars in 1152 bytes} [Thu May 26 15:16:13 2022] POST /identity/v3/projects => generated 371 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 15:16:13.172860 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fab9ac-fe59-4f22-b71a-f7ee361b424a admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:16:13.190400 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fab9ac-fe59-4f22-b71a-f7ee361b424a admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:16:13.261645 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721073bb-5492-4036-b44a-a849ad59ac32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:16:13.262426 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721073bb-5492-4036-b44a-a849ad59ac32 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:16:13.262773 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721073bb-5492-4036-b44a-a849ad59ac32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:16:13.263118 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721073bb-5492-4036-b44a-a849ad59ac32 admin admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:16:13.263919 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721073bb-5492-4036-b44a-a849ad59ac32 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:16:13.267240 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721073bb-5492-4036-b44a-a849ad59ac32 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:16:13.298190 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1215/2417] 10.222.0.46 () {60 vars in 1143 bytes} [Thu May 26 15:16:13 2022] POST /identity/v3/users => generated 286 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 15:16:13.302256 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5ed7f7-2073-4bd2-99d6-ed95510c21b9 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:16:13.312622 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5ed7f7-2073-4bd2-99d6-ed95510c21b9 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:16:13.317444 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b09076a-d769-412d-ba48-3ac71d68ba20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:16:13.318144 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b09076a-d769-412d-ba48-3ac71d68ba20 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:16:13.318424 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b09076a-d769-412d-ba48-3ac71d68ba20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:16:13.318702 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b09076a-d769-412d-ba48-3ac71d68ba20 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:16:13.319431 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b09076a-d769-412d-ba48-3ac71d68ba20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:16:13.321033 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b09076a-d769-412d-ba48-3ac71d68ba20 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:16:13.333719 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1203/2418] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 15:16:13 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 15:16:13.339184 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721073bb-5492-4036-b44a-a849ad59ac32 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:16:13.350452 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721073bb-5492-4036-b44a-a849ad59ac32 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:16:13.355716 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982b79fd-acc8-44e0-aab5-b62921de9bb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:16:13.356471 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982b79fd-acc8-44e0-aab5-b62921de9bb9 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:16:13.357539 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982b79fd-acc8-44e0-aab5-b62921de9bb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:16:13.357539 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982b79fd-acc8-44e0-aab5-b62921de9bb9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:16:13.358006 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982b79fd-acc8-44e0-aab5-b62921de9bb9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:16:13.359646 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982b79fd-acc8-44e0-aab5-b62921de9bb9 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:16:13.374980 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1216/2419] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 15:16:13 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 15:16:13.376086 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b09076a-d769-412d-ba48-3ac71d68ba20 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:16:13.387326 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b09076a-d769-412d-ba48-3ac71d68ba20 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:16:13.392195 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef9e916-7ba7-4646-8528-20aa79c88de9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:16:13.393095 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef9e916-7ba7-4646-8528-20aa79c88de9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:16:13.393383 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef9e916-7ba7-4646-8528-20aa79c88de9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:16:13.393703 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef9e916-7ba7-4646-8528-20aa79c88de9 admin admin] PATH_INFO: `/v3/projects/d639caecc6dc49558e374eb624a0fe91/users/da62bffb283b4fb8b4c01083e8c7cb87/roles/d4088d1845a24fbd9bd8975ce0b1d3d0` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:16:13.410731 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef9e916-7ba7-4646-8528-20aa79c88de9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d639caecc6dc49558e374eb624a0fe91, user_id=da62bffb283b4fb8b4c01083e8c7cb87, role_id=d4088d1845a24fbd9bd8975ce0b1d3d0)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:16:13.413640 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef9e916-7ba7-4646-8528-20aa79c88de9 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:16:13.431358 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1204/2420] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 15:16:13 2022] PUT /identity/v3/projects/d639caecc6dc49558e374eb624a0fe91/users/da62bffb283b4fb8b4c01083e8c7cb87/roles/d4088d1845a24fbd9bd8975ce0b1d3d0 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 15:16:13.442806 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd9fcfe-9169-484c-b34e-8c5488bc6769 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:16:13.443098 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd9fcfe-9169-484c-b34e-8c5488bc6769 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:16:13.443380 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd9fcfe-9169-484c-b34e-8c5488bc6769 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:16:13.473401 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-1cd9fcfe-9169-484c-b34e-8c5488bc6769 None None] MFA Rules not processed for user `da62bffb283b4fb8b4c01083e8c7cb87`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:16:13.475060 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-1cd9fcfe-9169-484c-b34e-8c5488bc6769 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:16:13.559645 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1217/2421] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:16:13 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:16:13.564401 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef9e916-7ba7-4646-8528-20aa79c88de9 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:16:13.636237 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef9e916-7ba7-4646-8528-20aa79c88de9 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:16:13.641158 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba02f332-8040-4a5e-b307-d39abcee6862 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:16:13.641854 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba02f332-8040-4a5e-b307-d39abcee6862 admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:16:13.642139 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba02f332-8040-4a5e-b307-d39abcee6862 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:16:13.642418 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba02f332-8040-4a5e-b307-d39abcee6862 admin admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:16:13.643129 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba02f332-8040-4a5e-b307-d39abcee6862 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:16:13.645064 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba02f332-8040-4a5e-b307-d39abcee6862 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:16:13.676822 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1205/2422] 10.222.0.46 () {60 vars in 1152 bytes} [Thu May 26 15:16:13 2022] POST /identity/v3/projects => generated 371 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 15:16:13.678211 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd9fcfe-9169-484c-b34e-8c5488bc6769 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:16:13.689352 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd9fcfe-9169-484c-b34e-8c5488bc6769 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:16:13.717401 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:16:13.750698 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c058452-149b-4b33-b8ed-c3c0c9619afe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:16:13.751395 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c058452-149b-4b33-b8ed-c3c0c9619afe admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:16:13.751684 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c058452-149b-4b33-b8ed-c3c0c9619afe admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:16:13.751968 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c058452-149b-4b33-b8ed-c3c0c9619afe admin admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:16:13.752665 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c058452-149b-4b33-b8ed-c3c0c9619afe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:16:13.754679 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c058452-149b-4b33-b8ed-c3c0c9619afe admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:16:13.777798 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1218/2423] 10.222.0.46 () {60 vars in 1143 bytes} [Thu May 26 15:16:13 2022] POST /identity/v3/users => generated 292 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 15:16:13.781981 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba02f332-8040-4a5e-b307-d39abcee6862 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:16:13.792319 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba02f332-8040-4a5e-b307-d39abcee6862 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:16:13.797172 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdd41ab-6461-4ec0-a818-452bf7414c86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:16:13.797978 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdd41ab-6461-4ec0-a818-452bf7414c86 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:16:13.798227 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdd41ab-6461-4ec0-a818-452bf7414c86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:16:13.798510 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdd41ab-6461-4ec0-a818-452bf7414c86 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:16:13.799239 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bdd41ab-6461-4ec0-a818-452bf7414c86 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:16:13.800852 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bdd41ab-6461-4ec0-a818-452bf7414c86 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:16:13.812265 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1206/2424] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 15:16:13 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 15:16:13.816618 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c058452-149b-4b33-b8ed-c3c0c9619afe admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:16:13.827393 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c058452-149b-4b33-b8ed-c3c0c9619afe admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:16:13.832386 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a4e1dc-cdb0-4700-93fb-22f072eef3f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:16:13.833287 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a4e1dc-cdb0-4700-93fb-22f072eef3f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:16:13.833613 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a4e1dc-cdb0-4700-93fb-22f072eef3f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:16:13.833897 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a4e1dc-cdb0-4700-93fb-22f072eef3f9 admin admin] PATH_INFO: `/v3/projects/29a409eaa55c4178bb5b41d7d6e37659/users/19b99c92a79b47c388f687f21de2c1cf/roles/a2e551773cbd4df084020676865b8222` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:16:13.849637 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a4e1dc-cdb0-4700-93fb-22f072eef3f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=29a409eaa55c4178bb5b41d7d6e37659, user_id=19b99c92a79b47c388f687f21de2c1cf, role_id=a2e551773cbd4df084020676865b8222)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:16:13.852352 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a4e1dc-cdb0-4700-93fb-22f072eef3f9 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:16:13.867790 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1219/2425] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 15:16:13 2022] PUT /identity/v3/projects/29a409eaa55c4178bb5b41d7d6e37659/users/19b99c92a79b47c388f687f21de2c1cf/roles/a2e551773cbd4df084020676865b8222 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:16:13.872358 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdd41ab-6461-4ec0-a818-452bf7414c86 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:16:13.883037 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdd41ab-6461-4ec0-a818-452bf7414c86 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:16:13.940866 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057c2a16-dd88-4b60-b2df-dc581a31e8d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:16:13.941540 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057c2a16-dd88-4b60-b2df-dc581a31e8d2 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:16:13.941823 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057c2a16-dd88-4b60-b2df-dc581a31e8d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:16:13.942097 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057c2a16-dd88-4b60-b2df-dc581a31e8d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:16:13.942815 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057c2a16-dd88-4b60-b2df-dc581a31e8d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:16:13.944411 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057c2a16-dd88-4b60-b2df-dc581a31e8d2 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:16:13.954845 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1207/2426] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 15:16:13 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 15:16:13.959080 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a4e1dc-cdb0-4700-93fb-22f072eef3f9 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:16:13.969894 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a4e1dc-cdb0-4700-93fb-22f072eef3f9 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:16:13.974577 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdc5b41-bdd8-48b1-9949-e4970c5e3662 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:16:13.975463 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdc5b41-bdd8-48b1-9949-e4970c5e3662 admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:16:13.975746 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdc5b41-bdd8-48b1-9949-e4970c5e3662 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:16:13.976024 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bdc5b41-bdd8-48b1-9949-e4970c5e3662 admin admin] PATH_INFO: `/v3/projects/29a409eaa55c4178bb5b41d7d6e37659/users/19b99c92a79b47c388f687f21de2c1cf/roles/a2e551773cbd4df084020676865b8222` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:16:13.978033 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bdc5b41-bdd8-48b1-9949-e4970c5e3662 admin admin] RBAC: Authorizing `identity:create_grant(project_id=29a409eaa55c4178bb5b41d7d6e37659, user_id=19b99c92a79b47c388f687f21de2c1cf, role_id=a2e551773cbd4df084020676865b8222)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:16:13.980683 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bdc5b41-bdd8-48b1-9949-e4970c5e3662 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:16:13.991417 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1220/2427] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 15:16:13 2022] PUT /identity/v3/projects/29a409eaa55c4178bb5b41d7d6e37659/users/19b99c92a79b47c388f687f21de2c1cf/roles/a2e551773cbd4df084020676865b8222 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:16:14.002347 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab62850-ca9d-44f0-9cfb-c7a8d8a1ee22 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:16:14.002645 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab62850-ca9d-44f0-9cfb-c7a8d8a1ee22 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:16:14.002930 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab62850-ca9d-44f0-9cfb-c7a8d8a1ee22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:16:14.047097 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-cab62850-ca9d-44f0-9cfb-c7a8d8a1ee22 None None] MFA Rules not processed for user `19b99c92a79b47c388f687f21de2c1cf`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:16:14.048473 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-cab62850-ca9d-44f0-9cfb-c7a8d8a1ee22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:16:14.065922 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:16:14.172712 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1208/2428] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:16:14 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:16:14.187214 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdc5b41-bdd8-48b1-9949-e4970c5e3662 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:16:14.189320 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-1bdc5b41-bdd8-48b1-9949-e4970c5e3662 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:16:14.264069 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bdc5b41-bdd8-48b1-9949-e4970c5e3662 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:16:14.268127 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd24b11-0835-4829-9954-3b9004f5be5d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5b5b35614a547d783e6e77ddb8c1c03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:16:14.268790 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd24b11-0835-4829-9954-3b9004f5be5d service cinder] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:16:14.269079 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd24b11-0835-4829-9954-3b9004f5be5d service cinder] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:16:14.269358 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd24b11-0835-4829-9954-3b9004f5be5d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:16:14.286107 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd24b11-0835-4829-9954-3b9004f5be5d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:16:14.287589 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd24b11-0835-4829-9954-3b9004f5be5d service cinder] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:16:14.298920 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1221/2429] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:16:14 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 15:16:14.385881 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab62850-ca9d-44f0-9cfb-c7a8d8a1ee22 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:16:14.387628 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-cab62850-ca9d-44f0-9cfb-c7a8d8a1ee22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:16:14.400391 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab62850-ca9d-44f0-9cfb-c7a8d8a1ee22 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:16:14.404031 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc02eeb-57d9-4713-b00d-f804110d1f98 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5b5b35614a547d783e6e77ddb8c1c03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:16:14.404679 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc02eeb-57d9-4713-b00d-f804110d1f98 service cinder] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:16:14.404965 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc02eeb-57d9-4713-b00d-f804110d1f98 service cinder] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:16:14.405243 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc02eeb-57d9-4713-b00d-f804110d1f98 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:16:14.468840 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc02eeb-57d9-4713-b00d-f804110d1f98 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:16:14.470214 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc02eeb-57d9-4713-b00d-f804110d1f98 service cinder] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:16:14.480407 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1209/2430] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:16:14 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 15:16:17.512944 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd24b11-0835-4829-9954-3b9004f5be5d service cinder] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:16:17.515170 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-8cd24b11-0835-4829-9954-3b9004f5be5d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:16:17.600444 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd24b11-0835-4829-9954-3b9004f5be5d service cinder] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:16:17.608432 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529bdf94-d3a4-4c93-95f2-c82ba815b894 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:16:17.609438 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529bdf94-d3a4-4c93-95f2-c82ba815b894 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:16:17.609891 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529bdf94-d3a4-4c93-95f2-c82ba815b894 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:16:17.610061 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529bdf94-d3a4-4c93-95f2-c82ba815b894 admin admin] PATH_INFO: `/v3/users/76b4411926974efab5b41d1f36df636e` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:16:17.612705 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529bdf94-d3a4-4c93-95f2-c82ba815b894 admin admin] RBAC: Authorizing `identity:delete_user(user_id=76b4411926974efab5b41d1f36df636e)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:16:17.614582 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-529bdf94-d3a4-4c93-95f2-c82ba815b894 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:16:17.650127 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=91,keystone:INSERT=5,keystone:DELETE=4 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:16:17.677885 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-529bdf94-d3a4-4c93-95f2-c82ba815b894 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '76b4411926974efab5b41d1f36df636e'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:16:17.688000 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-529bdf94-d3a4-4c93-95f2-c82ba815b894 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '76b4411926974efab5b41d1f36df636e'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:16:17.689002 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-529bdf94-d3a4-4c93-95f2-c82ba815b894 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '76b4411926974efab5b41d1f36df636e'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:16:17.706684 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-529bdf94-d3a4-4c93-95f2-c82ba815b894 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '76b4411926974efab5b41d1f36df636e'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:16:17.722087 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-529bdf94-d3a4-4c93-95f2-c82ba815b894 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '76b4411926974efab5b41d1f36df636e'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:16:17.730639 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1222/2431] 10.222.0.46 () {58 vars in 1223 bytes} [Thu May 26 15:16:17 2022] DELETE /identity/v3/users/76b4411926974efab5b41d1f36df636e => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:16:17.737735 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0e5081-7869-45c5-ab9b-c2f462fee67d None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:16:17.738526 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0e5081-7869-45c5-ab9b-c2f462fee67d None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:16:17.738709 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0e5081-7869-45c5-ab9b-c2f462fee67d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:16:17.759229 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-8c0e5081-7869-45c5-ab9b-c2f462fee67d None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:16:17.760525 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-8c0e5081-7869-45c5-ab9b-c2f462fee67d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:16:17.834973 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1210/2432] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:16:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:16:17.846327 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529bdf94-d3a4-4c93-95f2-c82ba815b894 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:16:17.849671 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-529bdf94-d3a4-4c93-95f2-c82ba815b894 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:16:17.921721 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529bdf94-d3a4-4c93-95f2-c82ba815b894 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:16:17.925566 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ee4495-daa5-4d76-a050-66e1a089e32b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0156a2f6f3794f479e76b85bdbbe5c26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:16:17.926311 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ee4495-daa5-4d76-a050-66e1a089e32b service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:16:17.926676 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ee4495-daa5-4d76-a050-66e1a089e32b service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:16:17.927012 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ee4495-daa5-4d76-a050-66e1a089e32b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:16:17.943502 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ee4495-daa5-4d76-a050-66e1a089e32b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:16:17.945207 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ee4495-daa5-4d76-a050-66e1a089e32b service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:16:17.963971 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1223/2433] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:16:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 15:16:18.088785 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0e5081-7869-45c5-ab9b-c2f462fee67d None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:16:18.090863 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-8c0e5081-7869-45c5-ab9b-c2f462fee67d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:16:18.105226 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0e5081-7869-45c5-ab9b-c2f462fee67d None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:16:18.110112 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d079e7e6-1aca-4bd8-988e-1e8e8172cdff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0156a2f6f3794f479e76b85bdbbe5c26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:16:18.110855 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d079e7e6-1aca-4bd8-988e-1e8e8172cdff service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:16:18.111132 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d079e7e6-1aca-4bd8-988e-1e8e8172cdff service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:16:18.111409 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d079e7e6-1aca-4bd8-988e-1e8e8172cdff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:16:18.113224 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-d079e7e6-1aca-4bd8-988e-1e8e8172cdff service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:16:18.128196 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d079e7e6-1aca-4bd8-988e-1e8e8172cdff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:16:18.129684 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d079e7e6-1aca-4bd8-988e-1e8e8172cdff service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:16:18.140385 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1211/2434] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:16:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 15:16:18.465601 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ee4495-daa5-4d76-a050-66e1a089e32b service neutron] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:16:18.467039 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-57ee4495-daa5-4d76-a050-66e1a089e32b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:16:18.481400 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ee4495-daa5-4d76-a050-66e1a089e32b service neutron] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:16:18.486249 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d3436f-7db6-4bc9-b64b-7d7716ce1d7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:16:18.487916 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d3436f-7db6-4bc9-b64b-7d7716ce1d7c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:16:18.487916 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d3436f-7db6-4bc9-b64b-7d7716ce1d7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:16:18.487916 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d3436f-7db6-4bc9-b64b-7d7716ce1d7c admin admin] PATH_INFO: `/v3/projects/85a93d9961c2466aae0e7dbd25e34876` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:16:18.489573 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d3436f-7db6-4bc9-b64b-7d7716ce1d7c admin admin] RBAC: Authorizing `identity:delete_project(project_id=85a93d9961c2466aae0e7dbd25e34876)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:16:18.491527 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43d3436f-7db6-4bc9-b64b-7d7716ce1d7c admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:16:18.564759 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-43d3436f-7db6-4bc9-b64b-7d7716ce1d7c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '85a93d9961c2466aae0e7dbd25e34876', 'request_id': 'req-43d3436f-7db6-4bc9-b64b-7d7716ce1d7c'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:16:18.575954 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-43d3436f-7db6-4bc9-b64b-7d7716ce1d7c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '85a93d9961c2466aae0e7dbd25e34876', 'request_id': 'req-43d3436f-7db6-4bc9-b64b-7d7716ce1d7c'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:16:18.583086 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-43d3436f-7db6-4bc9-b64b-7d7716ce1d7c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '85a93d9961c2466aae0e7dbd25e34876', 'request_id': 'req-43d3436f-7db6-4bc9-b64b-7d7716ce1d7c'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:16:18.595468 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-43d3436f-7db6-4bc9-b64b-7d7716ce1d7c admin admin] The token cache is being invalidate because project 85a93d9961c2466aae0e7dbd25e34876 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 15:16:18.595757 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-43d3436f-7db6-4bc9-b64b-7d7716ce1d7c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:16:18.596442 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-43d3436f-7db6-4bc9-b64b-7d7716ce1d7c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:16:18.601654 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1224/2435] 10.222.0.46 () {58 vars in 1232 bytes} [Thu May 26 15:16:18 2022] DELETE /identity/v3/projects/85a93d9961c2466aae0e7dbd25e34876 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:16:18.605488 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cd7e07-45a0-4c99-8fa4-2134478f4192 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:16:18.605659 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cd7e07-45a0-4c99-8fa4-2134478f4192 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:16:18.608562 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cd7e07-45a0-4c99-8fa4-2134478f4192 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:16:18.644306 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-03cd7e07-45a0-4c99-8fa4-2134478f4192 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:16:18.646008 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-03cd7e07-45a0-4c99-8fa4-2134478f4192 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:16:18.684438 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:16:18.722997 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1212/2436] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:16:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:16:18.731335 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25f56e6-4e2b-45af-82d4-b31ae199dfcd None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:16:18.731695 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25f56e6-4e2b-45af-82d4-b31ae199dfcd None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:16:18.732009 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25f56e6-4e2b-45af-82d4-b31ae199dfcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:16:18.757419 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-d25f56e6-4e2b-45af-82d4-b31ae199dfcd None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:16:18.758906 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-d25f56e6-4e2b-45af-82d4-b31ae199dfcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:16:18.768259 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1225/2437] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:16:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:16:18.772581 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cd7e07-45a0-4c99-8fa4-2134478f4192 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:16:18.784020 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cd7e07-45a0-4c99-8fa4-2134478f4192 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:16:18.791799 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81b4dc5-9a5d-4f7c-bc4d-e58570464d2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:16:18.792859 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81b4dc5-9a5d-4f7c-bc4d-e58570464d2d admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:16:18.793336 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81b4dc5-9a5d-4f7c-bc4d-e58570464d2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:16:18.793804 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81b4dc5-9a5d-4f7c-bc4d-e58570464d2d admin admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:16:18.795219 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81b4dc5-9a5d-4f7c-bc4d-e58570464d2d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 15:16:18.795751 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81b4dc5-9a5d-4f7c-bc4d-e58570464d2d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:16:18.798710 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81b4dc5-9a5d-4f7c-bc4d-e58570464d2d admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:16:18.825869 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1213/2438] 10.222.0.46 () {58 vars in 1152 bytes} [Thu May 26 15:16:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 15:16:18.831111 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25f56e6-4e2b-45af-82d4-b31ae199dfcd None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:16:18.842401 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25f56e6-4e2b-45af-82d4-b31ae199dfcd None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:16:18.848209 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115e4bf2-a352-4b09-8161-fc858959415f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:16:18.849263 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115e4bf2-a352-4b09-8161-fc858959415f admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:16:18.849263 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115e4bf2-a352-4b09-8161-fc858959415f admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:16:18.849585 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115e4bf2-a352-4b09-8161-fc858959415f admin admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:16:18.850145 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115e4bf2-a352-4b09-8161-fc858959415f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:16:18.852071 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115e4bf2-a352-4b09-8161-fc858959415f admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:16:18.878700 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:DELETE=13,keystone:SELECT=84,keystone:INSERT=3 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:16:18.888569 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1226/2439] 10.222.0.46 () {60 vars in 1152 bytes} [Thu May 26 15:16:18 2022] POST /identity/v3/projects => generated 373 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 15:16:18.892353 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81b4dc5-9a5d-4f7c-bc4d-e58570464d2d admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:16:18.904818 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81b4dc5-9a5d-4f7c-bc4d-e58570464d2d admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:16:18.972317 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1be7db0-1fdc-47fd-ae22-2e478bbb8aed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:16:18.973155 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1be7db0-1fdc-47fd-ae22-2e478bbb8aed admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:16:18.973478 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1be7db0-1fdc-47fd-ae22-2e478bbb8aed admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:16:18.973766 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1be7db0-1fdc-47fd-ae22-2e478bbb8aed admin admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:16:18.974458 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1be7db0-1fdc-47fd-ae22-2e478bbb8aed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:16:18.976457 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1be7db0-1fdc-47fd-ae22-2e478bbb8aed admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:16:19.000919 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1214/2440] 10.222.0.46 () {60 vars in 1143 bytes} [Thu May 26 15:16:18 2022] POST /identity/v3/users => generated 287 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 15:16:19.005024 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115e4bf2-a352-4b09-8161-fc858959415f admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:16:19.016115 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115e4bf2-a352-4b09-8161-fc858959415f admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:16:19.020704 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1100c1b-b6af-446e-bb17-521c91ced639 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:16:19.021377 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1100c1b-b6af-446e-bb17-521c91ced639 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:16:19.021701 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1100c1b-b6af-446e-bb17-521c91ced639 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:16:19.021982 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1100c1b-b6af-446e-bb17-521c91ced639 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:16:19.022795 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1100c1b-b6af-446e-bb17-521c91ced639 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:16:19.024402 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1100c1b-b6af-446e-bb17-521c91ced639 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:16:19.037736 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1227/2441] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 15:16:19 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 15:16:19.042286 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1be7db0-1fdc-47fd-ae22-2e478bbb8aed admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:16:19.054208 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1be7db0-1fdc-47fd-ae22-2e478bbb8aed admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:16:19.058891 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1f2b30-580b-4f73-8599-cc196b7b3f2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:16:19.059601 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1f2b30-580b-4f73-8599-cc196b7b3f2e admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:16:19.059839 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1f2b30-580b-4f73-8599-cc196b7b3f2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:16:19.060113 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1f2b30-580b-4f73-8599-cc196b7b3f2e admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:16:19.060836 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1f2b30-580b-4f73-8599-cc196b7b3f2e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:16:19.062492 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1f2b30-580b-4f73-8599-cc196b7b3f2e admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:16:19.072676 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1215/2442] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 15:16:19 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 15:16:19.076890 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1100c1b-b6af-446e-bb17-521c91ced639 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:16:19.086993 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1100c1b-b6af-446e-bb17-521c91ced639 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:16:19.092553 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b079101-e4f4-4987-a5fd-b89e6f05013b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:16:19.093443 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b079101-e4f4-4987-a5fd-b89e6f05013b admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:16:19.093767 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b079101-e4f4-4987-a5fd-b89e6f05013b admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:16:19.094153 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b079101-e4f4-4987-a5fd-b89e6f05013b admin admin] PATH_INFO: `/v3/projects/59fbca412c0346038d8b4a12129805f8/users/52fcc9ce57ff4afd99b29edceab1539c/roles/d4088d1845a24fbd9bd8975ce0b1d3d0` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:16:19.111313 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b079101-e4f4-4987-a5fd-b89e6f05013b admin admin] RBAC: Authorizing `identity:create_grant(project_id=59fbca412c0346038d8b4a12129805f8, user_id=52fcc9ce57ff4afd99b29edceab1539c, role_id=d4088d1845a24fbd9bd8975ce0b1d3d0)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:16:19.114037 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b079101-e4f4-4987-a5fd-b89e6f05013b admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:16:19.127940 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1228/2443] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 15:16:19 2022] PUT /identity/v3/projects/59fbca412c0346038d8b4a12129805f8/users/52fcc9ce57ff4afd99b29edceab1539c/roles/d4088d1845a24fbd9bd8975ce0b1d3d0 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 15:16:19.139209 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614ee61d-ace4-4d76-a654-eecc89956a9f None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:16:19.139507 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614ee61d-ace4-4d76-a654-eecc89956a9f None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:16:19.139792 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614ee61d-ace4-4d76-a654-eecc89956a9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:16:19.169668 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-614ee61d-ace4-4d76-a654-eecc89956a9f None None] MFA Rules not processed for user `52fcc9ce57ff4afd99b29edceab1539c`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:16:19.170867 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-614ee61d-ace4-4d76-a654-eecc89956a9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:16:19.252780 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1216/2444] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:16:19 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:16:19.264012 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b079101-e4f4-4987-a5fd-b89e6f05013b admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:16:19.265903 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-3b079101-e4f4-4987-a5fd-b89e6f05013b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:16:19.335222 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b079101-e4f4-4987-a5fd-b89e6f05013b admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:16:19.339837 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081500a3-84ac-4cf0-91cd-4bf0049fb615 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5b5b35614a547d783e6e77ddb8c1c03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:16:19.340598 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081500a3-84ac-4cf0-91cd-4bf0049fb615 service cinder] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:16:19.340889 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081500a3-84ac-4cf0-91cd-4bf0049fb615 service cinder] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:16:19.342105 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081500a3-84ac-4cf0-91cd-4bf0049fb615 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:16:19.355956 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081500a3-84ac-4cf0-91cd-4bf0049fb615 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:16:19.357948 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081500a3-84ac-4cf0-91cd-4bf0049fb615 service cinder] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:16:19.385640 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1229/2445] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:16:19 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 15:16:19.405185 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614ee61d-ace4-4d76-a654-eecc89956a9f None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:16:19.437788 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:16:19.473640 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614ee61d-ace4-4d76-a654-eecc89956a9f None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:16:19.478553 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249eab2e-f39f-4661-a7fc-b0de9627358c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:16:19.479666 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249eab2e-f39f-4661-a7fc-b0de9627358c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:16:19.479955 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249eab2e-f39f-4661-a7fc-b0de9627358c admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:16:19.480232 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249eab2e-f39f-4661-a7fc-b0de9627358c admin admin] PATH_INFO: `/v3/users/52fcc9ce57ff4afd99b29edceab1539c` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:16:19.481439 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249eab2e-f39f-4661-a7fc-b0de9627358c admin admin] RBAC: Authorizing `identity:delete_user(user_id=52fcc9ce57ff4afd99b29edceab1539c)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:16:19.483414 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249eab2e-f39f-4661-a7fc-b0de9627358c admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:16:19.538700 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-249eab2e-f39f-4661-a7fc-b0de9627358c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '52fcc9ce57ff4afd99b29edceab1539c'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:16:19.551941 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-249eab2e-f39f-4661-a7fc-b0de9627358c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '52fcc9ce57ff4afd99b29edceab1539c'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:16:19.552696 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-249eab2e-f39f-4661-a7fc-b0de9627358c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '52fcc9ce57ff4afd99b29edceab1539c'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:16:19.568032 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-249eab2e-f39f-4661-a7fc-b0de9627358c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '52fcc9ce57ff4afd99b29edceab1539c'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:16:19.576648 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-249eab2e-f39f-4661-a7fc-b0de9627358c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '52fcc9ce57ff4afd99b29edceab1539c'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:16:19.583880 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1217/2446] 10.222.0.46 () {58 vars in 1223 bytes} [Thu May 26 15:16:19 2022] DELETE /identity/v3/users/52fcc9ce57ff4afd99b29edceab1539c => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:16:19.588858 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8839053c-7283-40bb-84d7-b54f97ad11ef None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:16:19.589358 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8839053c-7283-40bb-84d7-b54f97ad11ef None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:16:19.589442 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8839053c-7283-40bb-84d7-b54f97ad11ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:16:19.608673 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-8839053c-7283-40bb-84d7-b54f97ad11ef None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:16:19.609940 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-8839053c-7283-40bb-84d7-b54f97ad11ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:16:19.676237 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1230/2447] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:16:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:16:19.686091 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249eab2e-f39f-4661-a7fc-b0de9627358c admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:16:19.687607 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-249eab2e-f39f-4661-a7fc-b0de9627358c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:16:19.774495 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249eab2e-f39f-4661-a7fc-b0de9627358c admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:16:19.785158 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd129ac-c1ed-408e-8bbb-94c1b4a1b9a6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0156a2f6f3794f479e76b85bdbbe5c26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:16:19.786033 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd129ac-c1ed-408e-8bbb-94c1b4a1b9a6 service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:16:19.786418 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd129ac-c1ed-408e-8bbb-94c1b4a1b9a6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:16:19.787192 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd129ac-c1ed-408e-8bbb-94c1b4a1b9a6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:16:19.797608 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd129ac-c1ed-408e-8bbb-94c1b4a1b9a6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:16:19.799312 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd129ac-c1ed-408e-8bbb-94c1b4a1b9a6 service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:16:19.811141 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1218/2448] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:16:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 15:16:20.288367 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8839053c-7283-40bb-84d7-b54f97ad11ef None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:16:20.290283 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-8839053c-7283-40bb-84d7-b54f97ad11ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:16:20.304514 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8839053c-7283-40bb-84d7-b54f97ad11ef None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:16:20.310543 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25ce676-c15f-40af-b06c-9f4877cfc8bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:16:20.311423 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25ce676-c15f-40af-b06c-9f4877cfc8bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:16:20.311705 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25ce676-c15f-40af-b06c-9f4877cfc8bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:16:20.312071 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25ce676-c15f-40af-b06c-9f4877cfc8bf admin admin] PATH_INFO: `/v3/projects/59fbca412c0346038d8b4a12129805f8` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:16:20.313193 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25ce676-c15f-40af-b06c-9f4877cfc8bf admin admin] RBAC: Authorizing `identity:delete_project(project_id=59fbca412c0346038d8b4a12129805f8)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:16:20.315195 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25ce676-c15f-40af-b06c-9f4877cfc8bf admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:16:20.365664 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-c25ce676-c15f-40af-b06c-9f4877cfc8bf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '59fbca412c0346038d8b4a12129805f8', 'request_id': 'req-c25ce676-c15f-40af-b06c-9f4877cfc8bf'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:16:20.374375 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-c25ce676-c15f-40af-b06c-9f4877cfc8bf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '59fbca412c0346038d8b4a12129805f8', 'request_id': 'req-c25ce676-c15f-40af-b06c-9f4877cfc8bf'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:16:20.379788 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:16:20.381277 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-c25ce676-c15f-40af-b06c-9f4877cfc8bf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '59fbca412c0346038d8b4a12129805f8', 'request_id': 'req-c25ce676-c15f-40af-b06c-9f4877cfc8bf'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:16:20.398379 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-c25ce676-c15f-40af-b06c-9f4877cfc8bf admin admin] The token cache is being invalidate because project 59fbca412c0346038d8b4a12129805f8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 15:16:20.398687 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-c25ce676-c15f-40af-b06c-9f4877cfc8bf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:16:20.399398 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-c25ce676-c15f-40af-b06c-9f4877cfc8bf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:16:20.403569 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1231/2449] 10.222.0.46 () {58 vars in 1232 bytes} [Thu May 26 15:16:20 2022] DELETE /identity/v3/projects/59fbca412c0346038d8b4a12129805f8 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:16:20.407617 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508cdd2c-7996-4f52-815b-e3c44a45af30 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:16:20.407911 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508cdd2c-7996-4f52-815b-e3c44a45af30 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:16:20.408217 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508cdd2c-7996-4f52-815b-e3c44a45af30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:16:20.431371 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-508cdd2c-7996-4f52-815b-e3c44a45af30 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:16:20.432763 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-508cdd2c-7996-4f52-815b-e3c44a45af30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:16:20.484285 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:16:20.507737 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1219/2450] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:16:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 15:16:20.516365 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2250ee1b-c495-4f12-aa67-198efc831ac9 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:16:20.516666 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2250ee1b-c495-4f12-aa67-198efc831ac9 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:16:20.516992 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2250ee1b-c495-4f12-aa67-198efc831ac9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:16:20.538296 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-2250ee1b-c495-4f12-aa67-198efc831ac9 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:16:20.539756 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-2250ee1b-c495-4f12-aa67-198efc831ac9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:16:20.548696 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1232/2451] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:16:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:16:20.555390 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508cdd2c-7996-4f52-815b-e3c44a45af30 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:16:20.569842 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508cdd2c-7996-4f52-815b-e3c44a45af30 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:16:20.577357 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188ad361-20a6-4a46-b485-30177f387235 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:16:20.578265 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188ad361-20a6-4a46-b485-30177f387235 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:16:20.578657 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188ad361-20a6-4a46-b485-30177f387235 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:16:20.579036 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188ad361-20a6-4a46-b485-30177f387235 admin admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:16:20.579892 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188ad361-20a6-4a46-b485-30177f387235 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 15:16:20.580308 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188ad361-20a6-4a46-b485-30177f387235 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:16:20.582155 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188ad361-20a6-4a46-b485-30177f387235 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:16:20.602158 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1220/2452] 10.222.0.46 () {58 vars in 1152 bytes} [Thu May 26 15:16:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 15:16:20.603512 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2250ee1b-c495-4f12-aa67-198efc831ac9 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:16:20.613913 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2250ee1b-c495-4f12-aa67-198efc831ac9 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:16:20.630192 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a458a2c-4dc8-4873-b302-82df66893e83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:16:20.631133 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a458a2c-4dc8-4873-b302-82df66893e83 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:16:20.632767 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a458a2c-4dc8-4873-b302-82df66893e83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:16:20.633333 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a458a2c-4dc8-4873-b302-82df66893e83 admin admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:16:20.634327 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a458a2c-4dc8-4873-b302-82df66893e83 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:16:20.637706 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a458a2c-4dc8-4873-b302-82df66893e83 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:16:20.670155 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1233/2453] 10.222.0.46 () {60 vars in 1152 bytes} [Thu May 26 15:16:20 2022] POST /identity/v3/projects => generated 385 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 15:16:20.674907 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188ad361-20a6-4a46-b485-30177f387235 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:16:20.686570 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188ad361-20a6-4a46-b485-30177f387235 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:16:20.754208 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b857a48-d0bb-4d79-904c-1133347fe90e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:16:20.755031 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b857a48-d0bb-4d79-904c-1133347fe90e admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:16:20.755319 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b857a48-d0bb-4d79-904c-1133347fe90e admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:16:20.755598 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b857a48-d0bb-4d79-904c-1133347fe90e admin admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:16:20.756292 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b857a48-d0bb-4d79-904c-1133347fe90e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:16:20.758303 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b857a48-d0bb-4d79-904c-1133347fe90e admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:16:20.785169 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1221/2454] 10.222.0.46 () {60 vars in 1143 bytes} [Thu May 26 15:16:20 2022] POST /identity/v3/users => generated 293 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 15:16:20.789442 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a458a2c-4dc8-4873-b302-82df66893e83 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:16:20.801014 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a458a2c-4dc8-4873-b302-82df66893e83 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:16:20.805916 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a17dc3-53e7-43e4-b0c9-e9a9e93f3f93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:16:20.807157 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a17dc3-53e7-43e4-b0c9-e9a9e93f3f93 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:16:20.807458 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a17dc3-53e7-43e4-b0c9-e9a9e93f3f93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:16:20.807736 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a17dc3-53e7-43e4-b0c9-e9a9e93f3f93 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:16:20.808467 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a17dc3-53e7-43e4-b0c9-e9a9e93f3f93 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:16:20.810122 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a17dc3-53e7-43e4-b0c9-e9a9e93f3f93 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:16:20.821045 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1234/2455] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 15:16:20 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 15:16:20.825823 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b857a48-d0bb-4d79-904c-1133347fe90e admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:16:20.836720 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b857a48-d0bb-4d79-904c-1133347fe90e admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:16:20.842345 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4818a67-a89e-41fd-b398-2a086a96f8db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:16:20.843117 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4818a67-a89e-41fd-b398-2a086a96f8db admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:16:20.843521 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4818a67-a89e-41fd-b398-2a086a96f8db admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:16:20.843904 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4818a67-a89e-41fd-b398-2a086a96f8db admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:16:20.844797 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4818a67-a89e-41fd-b398-2a086a96f8db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:16:20.846604 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4818a67-a89e-41fd-b398-2a086a96f8db admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:16:20.858490 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1222/2456] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 15:16:20 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 15:16:20.862932 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a17dc3-53e7-43e4-b0c9-e9a9e93f3f93 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:16:20.874568 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a17dc3-53e7-43e4-b0c9-e9a9e93f3f93 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:16:20.879914 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec8501c-fd0c-4b98-928f-86abf6cc0f59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:16:20.880837 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec8501c-fd0c-4b98-928f-86abf6cc0f59 admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:16:20.881122 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec8501c-fd0c-4b98-928f-86abf6cc0f59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:16:20.881401 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec8501c-fd0c-4b98-928f-86abf6cc0f59 admin admin] PATH_INFO: `/v3/projects/fb7e58e0c6954bb5ae0fafee4c7a6724/users/e6e2622df88f4e1b9b81097d6e558f21/roles/d4088d1845a24fbd9bd8975ce0b1d3d0` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:16:20.896564 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec8501c-fd0c-4b98-928f-86abf6cc0f59 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fb7e58e0c6954bb5ae0fafee4c7a6724, user_id=e6e2622df88f4e1b9b81097d6e558f21, role_id=d4088d1845a24fbd9bd8975ce0b1d3d0)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:16:20.899277 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec8501c-fd0c-4b98-928f-86abf6cc0f59 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:16:20.914181 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1235/2457] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 15:16:20 2022] PUT /identity/v3/projects/fb7e58e0c6954bb5ae0fafee4c7a6724/users/e6e2622df88f4e1b9b81097d6e558f21/roles/d4088d1845a24fbd9bd8975ce0b1d3d0 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:16:20.920298 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c832d5-732e-4a94-a1c7-75827a46402b None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:16:20.920511 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c832d5-732e-4a94-a1c7-75827a46402b None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:16:20.920796 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c832d5-732e-4a94-a1c7-75827a46402b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:16:20.942031 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-21c832d5-732e-4a94-a1c7-75827a46402b None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:16:20.943991 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-21c832d5-732e-4a94-a1c7-75827a46402b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:16:21.024673 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1223/2458] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:16:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:16:21.035667 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec8501c-fd0c-4b98-928f-86abf6cc0f59 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:16:21.037399 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-3ec8501c-fd0c-4b98-928f-86abf6cc0f59 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:16:21.131281 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec8501c-fd0c-4b98-928f-86abf6cc0f59 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:16:21.135083 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22f7480-9ed4-4ca3-93de-f681affa2097 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0156a2f6f3794f479e76b85bdbbe5c26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:16:21.135739 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22f7480-9ed4-4ca3-93de-f681affa2097 service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:16:21.136009 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22f7480-9ed4-4ca3-93de-f681affa2097 service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:16:21.136283 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22f7480-9ed4-4ca3-93de-f681affa2097 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:16:21.151855 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22f7480-9ed4-4ca3-93de-f681affa2097 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:16:21.153940 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22f7480-9ed4-4ca3-93de-f681affa2097 service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:16:21.169334 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1236/2459] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:16:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 15:16:21.669305 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c832d5-732e-4a94-a1c7-75827a46402b None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:16:21.671297 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-21c832d5-732e-4a94-a1c7-75827a46402b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:16:21.777388 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:16:21.793136 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22f7480-9ed4-4ca3-93de-f681affa2097 service neutron] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:16:21.795795 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-e22f7480-9ed4-4ca3-93de-f681affa2097 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:16:21.803991 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c832d5-732e-4a94-a1c7-75827a46402b None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:16:21.816082 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22f7480-9ed4-4ca3-93de-f681affa2097 service neutron] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:16:21.816901 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b63de2-8cab-4492-bcfc-5fb0ed13edeb service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf55a00e3b084aa3b22c25a90e6a773c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:16:21.818476 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b63de2-8cab-4492-bcfc-5fb0ed13edeb service placement] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:16:21.819008 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b63de2-8cab-4492-bcfc-5fb0ed13edeb service placement] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:16:21.819532 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b63de2-8cab-4492-bcfc-5fb0ed13edeb service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:16:21.824036 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-f9b63de2-8cab-4492-bcfc-5fb0ed13edeb service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:16:21.826362 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8433eb0-e7a5-4e14-9f37-20bdc03fa8fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:16:21.827696 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8433eb0-e7a5-4e14-9f37-20bdc03fa8fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:16:21.828234 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8433eb0-e7a5-4e14-9f37-20bdc03fa8fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:16:21.828690 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8433eb0-e7a5-4e14-9f37-20bdc03fa8fe admin admin] PATH_INFO: `/v3/users/64aeada36ebe460e9969f1f048651e6d` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:16:21.830213 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8433eb0-e7a5-4e14-9f37-20bdc03fa8fe admin admin] RBAC: Authorizing `identity:delete_user(user_id=64aeada36ebe460e9969f1f048651e6d)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:16:21.832560 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8433eb0-e7a5-4e14-9f37-20bdc03fa8fe admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:16:21.851433 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b63de2-8cab-4492-bcfc-5fb0ed13edeb service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:16:21.857848 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b63de2-8cab-4492-bcfc-5fb0ed13edeb service placement] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:16:21.878964 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1224/2460] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:16:21 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 15:16:21.902456 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-f8433eb0-e7a5-4e14-9f37-20bdc03fa8fe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '64aeada36ebe460e9969f1f048651e6d'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:16:21.920842 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-f8433eb0-e7a5-4e14-9f37-20bdc03fa8fe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '64aeada36ebe460e9969f1f048651e6d'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:16:21.922424 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-f8433eb0-e7a5-4e14-9f37-20bdc03fa8fe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '64aeada36ebe460e9969f1f048651e6d'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:16:21.937020 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-f8433eb0-e7a5-4e14-9f37-20bdc03fa8fe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '64aeada36ebe460e9969f1f048651e6d'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:16:21.950085 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-f8433eb0-e7a5-4e14-9f37-20bdc03fa8fe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '64aeada36ebe460e9969f1f048651e6d'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:16:21.977376 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1237/2461] 10.222.0.46 () {58 vars in 1223 bytes} [Thu May 26 15:16:21 2022] DELETE /identity/v3/users/64aeada36ebe460e9969f1f048651e6d => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 15:16:22.244620 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b63de2-8cab-4492-bcfc-5fb0ed13edeb service placement] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:16:22.262181 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b63de2-8cab-4492-bcfc-5fb0ed13edeb service placement] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:16:22.340679 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7b5d8c-2fe0-4688-ab1c-00a3a3536373 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:16:22.341599 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7b5d8c-2fe0-4688-ab1c-00a3a3536373 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:16:22.341887 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7b5d8c-2fe0-4688-ab1c-00a3a3536373 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:16:22.342364 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7b5d8c-2fe0-4688-ab1c-00a3a3536373 admin admin] PATH_INFO: `/v3/projects/d732418a4b2247e59ed96898b21cf6b1` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:16:22.343812 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7b5d8c-2fe0-4688-ab1c-00a3a3536373 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d732418a4b2247e59ed96898b21cf6b1)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:16:22.346128 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7b5d8c-2fe0-4688-ab1c-00a3a3536373 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:16:22.414821 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-3e7b5d8c-2fe0-4688-ab1c-00a3a3536373 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd732418a4b2247e59ed96898b21cf6b1', 'request_id': 'req-3e7b5d8c-2fe0-4688-ab1c-00a3a3536373'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:16:22.415791 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-3e7b5d8c-2fe0-4688-ab1c-00a3a3536373 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd732418a4b2247e59ed96898b21cf6b1', 'request_id': 'req-3e7b5d8c-2fe0-4688-ab1c-00a3a3536373'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:16:22.423088 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-3e7b5d8c-2fe0-4688-ab1c-00a3a3536373 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd732418a4b2247e59ed96898b21cf6b1', 'request_id': 'req-3e7b5d8c-2fe0-4688-ab1c-00a3a3536373'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:16:22.436906 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-3e7b5d8c-2fe0-4688-ab1c-00a3a3536373 admin admin] The token cache is being invalidate because project d732418a4b2247e59ed96898b21cf6b1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 15:16:22.437204 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-3e7b5d8c-2fe0-4688-ab1c-00a3a3536373 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:16:22.437867 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-3e7b5d8c-2fe0-4688-ab1c-00a3a3536373 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:16:22.439459 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1225/2462] 10.222.0.46 () {58 vars in 1232 bytes} [Thu May 26 15:16:22 2022] DELETE /identity/v3/projects/d732418a4b2247e59ed96898b21cf6b1 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:16:22.444538 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7006fda7-2983-4cf4-9cee-f443df52cae1 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:16:22.444905 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7006fda7-2983-4cf4-9cee-f443df52cae1 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:16:22.445257 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7006fda7-2983-4cf4-9cee-f443df52cae1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:16:22.452494 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:16:22.467844 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-7006fda7-2983-4cf4-9cee-f443df52cae1 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:16:22.469021 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-7006fda7-2983-4cf4-9cee-f443df52cae1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:16:22.544106 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1238/2463] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:16:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:16:22.551984 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9677e609-61fa-4844-b258-ce1efa7046b3 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:16:22.552553 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9677e609-61fa-4844-b258-ce1efa7046b3 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:16:22.552553 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9677e609-61fa-4844-b258-ce1efa7046b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:16:22.572063 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-9677e609-61fa-4844-b258-ce1efa7046b3 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:16:22.573336 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-9677e609-61fa-4844-b258-ce1efa7046b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:16:22.586641 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1226/2464] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:16:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:16:22.592689 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7006fda7-2983-4cf4-9cee-f443df52cae1 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:16:22.609051 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7006fda7-2983-4cf4-9cee-f443df52cae1 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:16:22.617597 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa421a5-3e59-4fe2-8338-6d1a54868a27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:16:22.618713 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa421a5-3e59-4fe2-8338-6d1a54868a27 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:16:22.620330 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa421a5-3e59-4fe2-8338-6d1a54868a27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:16:22.625795 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa421a5-3e59-4fe2-8338-6d1a54868a27 admin admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:16:22.625795 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa421a5-3e59-4fe2-8338-6d1a54868a27 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 15:16:22.625795 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa421a5-3e59-4fe2-8338-6d1a54868a27 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:16:22.626403 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa421a5-3e59-4fe2-8338-6d1a54868a27 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:16:22.657546 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1239/2465] 10.222.0.46 () {58 vars in 1152 bytes} [Thu May 26 15:16:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 15:16:22.659566 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9677e609-61fa-4844-b258-ce1efa7046b3 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:16:22.670892 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9677e609-61fa-4844-b258-ce1efa7046b3 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:16:22.683232 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73912620-f502-47a5-a2fd-8c8b7bf6cc87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:16:22.684497 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73912620-f502-47a5-a2fd-8c8b7bf6cc87 admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:16:22.685698 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73912620-f502-47a5-a2fd-8c8b7bf6cc87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:16:22.686289 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73912620-f502-47a5-a2fd-8c8b7bf6cc87 admin admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:16:22.687312 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73912620-f502-47a5-a2fd-8c8b7bf6cc87 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:16:22.690056 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73912620-f502-47a5-a2fd-8c8b7bf6cc87 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:16:22.751682 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1227/2466] 10.222.0.46 () {60 vars in 1152 bytes} [Thu May 26 15:16:22 2022] POST /identity/v3/projects => generated 373 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 15:16:22.754685 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa421a5-3e59-4fe2-8338-6d1a54868a27 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:16:22.772565 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa421a5-3e59-4fe2-8338-6d1a54868a27 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:16:22.856323 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0449ad9-0c7e-41f5-9d9a-27d1ad57030b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:16:22.857806 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0449ad9-0c7e-41f5-9d9a-27d1ad57030b admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:16:22.858498 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0449ad9-0c7e-41f5-9d9a-27d1ad57030b admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:16:22.859122 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0449ad9-0c7e-41f5-9d9a-27d1ad57030b admin admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:16:22.860510 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0449ad9-0c7e-41f5-9d9a-27d1ad57030b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:16:22.864526 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0449ad9-0c7e-41f5-9d9a-27d1ad57030b admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:16:22.890932 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1240/2467] 10.222.0.46 () {60 vars in 1143 bytes} [Thu May 26 15:16:22 2022] POST /identity/v3/users => generated 287 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 15:16:22.895478 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73912620-f502-47a5-a2fd-8c8b7bf6cc87 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:16:22.908653 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73912620-f502-47a5-a2fd-8c8b7bf6cc87 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:16:22.916471 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12651604-c11a-4591-ba00-67fa300fdb2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:16:22.918104 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12651604-c11a-4591-ba00-67fa300fdb2b admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:16:22.918481 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12651604-c11a-4591-ba00-67fa300fdb2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:16:22.919497 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12651604-c11a-4591-ba00-67fa300fdb2b admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:16:22.920713 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12651604-c11a-4591-ba00-67fa300fdb2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:16:22.923628 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12651604-c11a-4591-ba00-67fa300fdb2b admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:16:22.942451 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1228/2468] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 15:16:22 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 15:16:22.945957 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0449ad9-0c7e-41f5-9d9a-27d1ad57030b admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:16:22.964511 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0449ad9-0c7e-41f5-9d9a-27d1ad57030b admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:16:22.971860 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538614dd-94d8-46d4-95fb-1248e22d1817 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:16:22.972643 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538614dd-94d8-46d4-95fb-1248e22d1817 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:16:22.973021 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538614dd-94d8-46d4-95fb-1248e22d1817 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:16:22.973509 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538614dd-94d8-46d4-95fb-1248e22d1817 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:16:22.974286 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538614dd-94d8-46d4-95fb-1248e22d1817 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:16:22.976178 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538614dd-94d8-46d4-95fb-1248e22d1817 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:16:22.990240 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1241/2469] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 15:16:22 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 15:16:22.995764 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12651604-c11a-4591-ba00-67fa300fdb2b admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:16:23.008019 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12651604-c11a-4591-ba00-67fa300fdb2b admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:16:23.014669 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed846765-3db6-43c4-8793-fe2d908cf005 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:16:23.015603 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed846765-3db6-43c4-8793-fe2d908cf005 admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:16:23.015891 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed846765-3db6-43c4-8793-fe2d908cf005 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:16:23.016171 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed846765-3db6-43c4-8793-fe2d908cf005 admin admin] PATH_INFO: `/v3/projects/8156746b96cf4f3dbb904e3de3353ae8/users/c4660d71b63440d5b19ef371d4deee50/roles/d4088d1845a24fbd9bd8975ce0b1d3d0` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:16:23.043389 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed846765-3db6-43c4-8793-fe2d908cf005 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8156746b96cf4f3dbb904e3de3353ae8, user_id=c4660d71b63440d5b19ef371d4deee50, role_id=d4088d1845a24fbd9bd8975ce0b1d3d0)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:16:23.046261 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed846765-3db6-43c4-8793-fe2d908cf005 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:16:23.065551 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1229/2470] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 15:16:22 2022] PUT /identity/v3/projects/8156746b96cf4f3dbb904e3de3353ae8/users/c4660d71b63440d5b19ef371d4deee50/roles/d4088d1845a24fbd9bd8975ce0b1d3d0 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:16:23.077106 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd2dcbb-9aa0-46e3-b2e3-1ec103e0f752 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:16:23.077482 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd2dcbb-9aa0-46e3-b2e3-1ec103e0f752 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:16:23.077794 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd2dcbb-9aa0-46e3-b2e3-1ec103e0f752 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:16:23.121861 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-4bd2dcbb-9aa0-46e3-b2e3-1ec103e0f752 None None] MFA Rules not processed for user `c4660d71b63440d5b19ef371d4deee50`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:16:23.123235 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-4bd2dcbb-9aa0-46e3-b2e3-1ec103e0f752 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:16:23.226411 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:16:23.231900 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1242/2471] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:16:23 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:16:23.236242 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed846765-3db6-43c4-8793-fe2d908cf005 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:16:23.263231 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:16:23.339036 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed846765-3db6-43c4-8793-fe2d908cf005 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:16:23.343963 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e283692e-4296-45a8-aa56-e6a30f36eba0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:16:23.345170 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e283692e-4296-45a8-aa56-e6a30f36eba0 admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:16:23.345170 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e283692e-4296-45a8-aa56-e6a30f36eba0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:16:23.345634 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e283692e-4296-45a8-aa56-e6a30f36eba0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:16:23.347151 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e283692e-4296-45a8-aa56-e6a30f36eba0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:16:23.348504 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e283692e-4296-45a8-aa56-e6a30f36eba0 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:16:23.386206 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1230/2472] 10.222.0.46 () {60 vars in 1152 bytes} [Thu May 26 15:16:23 2022] POST /identity/v3/projects => generated 373 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 15:16:23.388599 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd2dcbb-9aa0-46e3-b2e3-1ec103e0f752 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:16:23.408060 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd2dcbb-9aa0-46e3-b2e3-1ec103e0f752 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:16:23.562765 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78db2056-0801-468c-ba3a-9798431d4d81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:16:23.563468 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78db2056-0801-468c-ba3a-9798431d4d81 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:16:23.563754 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78db2056-0801-468c-ba3a-9798431d4d81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:16:23.564032 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78db2056-0801-468c-ba3a-9798431d4d81 admin admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:16:23.564834 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78db2056-0801-468c-ba3a-9798431d4d81 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:16:23.567154 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78db2056-0801-468c-ba3a-9798431d4d81 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:16:23.599231 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1243/2473] 10.222.0.46 () {60 vars in 1143 bytes} [Thu May 26 15:16:23 2022] POST /identity/v3/users => generated 293 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 15:16:23.607983 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e283692e-4296-45a8-aa56-e6a30f36eba0 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:16:23.620809 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e283692e-4296-45a8-aa56-e6a30f36eba0 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:16:23.625969 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66706b3d-2fdf-469e-b918-77274e960fa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:16:23.626689 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66706b3d-2fdf-469e-b918-77274e960fa3 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:16:23.626977 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66706b3d-2fdf-469e-b918-77274e960fa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:16:23.627258 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66706b3d-2fdf-469e-b918-77274e960fa3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:16:23.628198 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66706b3d-2fdf-469e-b918-77274e960fa3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:16:23.631929 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66706b3d-2fdf-469e-b918-77274e960fa3 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:16:23.655526 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1231/2474] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 15:16:23 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 15:16:23.660394 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78db2056-0801-468c-ba3a-9798431d4d81 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:16:23.679220 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78db2056-0801-468c-ba3a-9798431d4d81 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:16:23.686130 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304536f8-5bf2-4eae-9236-0edbb80629c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:16:23.687153 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304536f8-5bf2-4eae-9236-0edbb80629c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:16:23.687650 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304536f8-5bf2-4eae-9236-0edbb80629c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:16:23.688009 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304536f8-5bf2-4eae-9236-0edbb80629c6 admin admin] PATH_INFO: `/v3/projects/f3d3cb2010644b588a9d699c1ba40d72/users/24491c222ac74750a09e3acccaf24bd9/roles/a2e551773cbd4df084020676865b8222` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:16:23.709990 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304536f8-5bf2-4eae-9236-0edbb80629c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f3d3cb2010644b588a9d699c1ba40d72, user_id=24491c222ac74750a09e3acccaf24bd9, role_id=a2e551773cbd4df084020676865b8222)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:16:23.714204 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304536f8-5bf2-4eae-9236-0edbb80629c6 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:16:23.752948 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1244/2475] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 15:16:23 2022] PUT /identity/v3/projects/f3d3cb2010644b588a9d699c1ba40d72/users/24491c222ac74750a09e3acccaf24bd9/roles/a2e551773cbd4df084020676865b8222 => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:16:23.754947 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66706b3d-2fdf-469e-b918-77274e960fa3 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:16:23.784363 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66706b3d-2fdf-469e-b918-77274e960fa3 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:16:23.887343 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f888e1a1-6fe4-4bb5-ae4f-4573e70199b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:16:23.888510 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f888e1a1-6fe4-4bb5-ae4f-4573e70199b4 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:16:23.889340 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f888e1a1-6fe4-4bb5-ae4f-4573e70199b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:16:23.889591 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f888e1a1-6fe4-4bb5-ae4f-4573e70199b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:16:23.891056 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f888e1a1-6fe4-4bb5-ae4f-4573e70199b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:16:23.894584 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f888e1a1-6fe4-4bb5-ae4f-4573e70199b4 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:16:23.910131 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1232/2476] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 15:16:23 2022] GET /identity/v3/roles => generated 1420 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 15:16:23.915719 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304536f8-5bf2-4eae-9236-0edbb80629c6 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:16:23.939360 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304536f8-5bf2-4eae-9236-0edbb80629c6 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:16:23.946466 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac529bee-72ab-490c-9da0-4d7371660df0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:16:23.948019 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac529bee-72ab-490c-9da0-4d7371660df0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:16:23.948378 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac529bee-72ab-490c-9da0-4d7371660df0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:16:23.948732 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac529bee-72ab-490c-9da0-4d7371660df0 admin admin] PATH_INFO: `/v3/projects/f3d3cb2010644b588a9d699c1ba40d72/users/24491c222ac74750a09e3acccaf24bd9/roles/a2e551773cbd4df084020676865b8222` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:16:23.952477 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac529bee-72ab-490c-9da0-4d7371660df0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f3d3cb2010644b588a9d699c1ba40d72, user_id=24491c222ac74750a09e3acccaf24bd9, role_id=a2e551773cbd4df084020676865b8222)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:16:23.958662 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac529bee-72ab-490c-9da0-4d7371660df0 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:16:23.978529 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1245/2477] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 15:16:23 2022] PUT /identity/v3/projects/f3d3cb2010644b588a9d699c1ba40d72/users/24491c222ac74750a09e3acccaf24bd9/roles/a2e551773cbd4df084020676865b8222 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:16:23.995219 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd41ef9-1a2a-40e7-81f4-f88d7bd23b06 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:16:23.995629 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd41ef9-1a2a-40e7-81f4-f88d7bd23b06 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:16:23.996094 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd41ef9-1a2a-40e7-81f4-f88d7bd23b06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:16:24.029366 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-3cd41ef9-1a2a-40e7-81f4-f88d7bd23b06 None None] MFA Rules not processed for user `24491c222ac74750a09e3acccaf24bd9`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:16:24.030631 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-3cd41ef9-1a2a-40e7-81f4-f88d7bd23b06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:16:24.128222 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1233/2478] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:16:23 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:16:24.141382 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac529bee-72ab-490c-9da0-4d7371660df0 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:16:24.143373 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-ac529bee-72ab-490c-9da0-4d7371660df0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:16:24.213950 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac529bee-72ab-490c-9da0-4d7371660df0 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:16:24.217500 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4961ef-cf15-4e69-9ff4-06ec46536715 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5b5b35614a547d783e6e77ddb8c1c03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:16:24.218279 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4961ef-cf15-4e69-9ff4-06ec46536715 service cinder] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:16:24.218756 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4961ef-cf15-4e69-9ff4-06ec46536715 service cinder] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:16:24.219144 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4961ef-cf15-4e69-9ff4-06ec46536715 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:16:24.231002 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4961ef-cf15-4e69-9ff4-06ec46536715 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:16:24.232638 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4961ef-cf15-4e69-9ff4-06ec46536715 service cinder] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:16:24.250362 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1246/2479] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:16:24 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 15:16:24.844058 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd41ef9-1a2a-40e7-81f4-f88d7bd23b06 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:16:24.857110 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:16:24.990284 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd41ef9-1a2a-40e7-81f4-f88d7bd23b06 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:16:25.005239 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee556728-32ed-4019-a2e7-15b49ff05577 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:16:25.006723 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee556728-32ed-4019-a2e7-15b49ff05577 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:16:25.007163 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee556728-32ed-4019-a2e7-15b49ff05577 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:16:25.007600 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee556728-32ed-4019-a2e7-15b49ff05577 admin admin] PATH_INFO: `/v3/users/c4660d71b63440d5b19ef371d4deee50` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:16:25.009178 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee556728-32ed-4019-a2e7-15b49ff05577 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c4660d71b63440d5b19ef371d4deee50)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:16:25.012446 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee556728-32ed-4019-a2e7-15b49ff05577 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:16:25.085011 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-ee556728-32ed-4019-a2e7-15b49ff05577 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c4660d71b63440d5b19ef371d4deee50'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:16:25.099534 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-ee556728-32ed-4019-a2e7-15b49ff05577 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c4660d71b63440d5b19ef371d4deee50'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:16:25.100756 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-ee556728-32ed-4019-a2e7-15b49ff05577 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c4660d71b63440d5b19ef371d4deee50'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:16:25.117106 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-ee556728-32ed-4019-a2e7-15b49ff05577 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c4660d71b63440d5b19ef371d4deee50'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:16:25.128044 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-ee556728-32ed-4019-a2e7-15b49ff05577 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c4660d71b63440d5b19ef371d4deee50'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:16:25.142409 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1234/2480] 10.222.0.46 () {58 vars in 1223 bytes} [Thu May 26 15:16:24 2022] DELETE /identity/v3/users/c4660d71b63440d5b19ef371d4deee50 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:16:25.148502 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7261c94-1c98-4b58-98a7-7373b592dffc None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:16:25.149044 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7261c94-1c98-4b58-98a7-7373b592dffc None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:16:25.149226 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7261c94-1c98-4b58-98a7-7373b592dffc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:16:25.173481 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-b7261c94-1c98-4b58-98a7-7373b592dffc None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:16:25.175310 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-b7261c94-1c98-4b58-98a7-7373b592dffc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:16:25.254136 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:16:25.277739 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1247/2481] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:16:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 15:16:25.298520 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee556728-32ed-4019-a2e7-15b49ff05577 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:16:25.300386 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-ee556728-32ed-4019-a2e7-15b49ff05577 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:16:25.388095 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee556728-32ed-4019-a2e7-15b49ff05577 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:16:25.393133 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0e83af-d2a7-4acd-b8cf-7a6aebea5ef1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0156a2f6f3794f479e76b85bdbbe5c26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:16:25.393759 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0e83af-d2a7-4acd-b8cf-7a6aebea5ef1 service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:16:25.394250 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0e83af-d2a7-4acd-b8cf-7a6aebea5ef1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:16:25.394758 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0e83af-d2a7-4acd-b8cf-7a6aebea5ef1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:16:25.408994 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0e83af-d2a7-4acd-b8cf-7a6aebea5ef1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:16:25.411002 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0e83af-d2a7-4acd-b8cf-7a6aebea5ef1 service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:16:25.430315 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1235/2482] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:16:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 15:16:26.028011 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7261c94-1c98-4b58-98a7-7373b592dffc None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:16:26.030669 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-b7261c94-1c98-4b58-98a7-7373b592dffc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:16:26.057998 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7261c94-1c98-4b58-98a7-7373b592dffc None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:16:26.067787 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1386536-471d-4597-b120-88e83eb3d251 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:16:26.069116 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1386536-471d-4597-b120-88e83eb3d251 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:16:26.069406 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1386536-471d-4597-b120-88e83eb3d251 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:16:26.069720 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1386536-471d-4597-b120-88e83eb3d251 admin admin] PATH_INFO: `/v3/projects/8156746b96cf4f3dbb904e3de3353ae8` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:16:26.071061 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1386536-471d-4597-b120-88e83eb3d251 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8156746b96cf4f3dbb904e3de3353ae8)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:16:26.074510 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1386536-471d-4597-b120-88e83eb3d251 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:16:26.155286 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-a1386536-471d-4597-b120-88e83eb3d251 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8156746b96cf4f3dbb904e3de3353ae8', 'request_id': 'req-a1386536-471d-4597-b120-88e83eb3d251'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:16:26.167327 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-a1386536-471d-4597-b120-88e83eb3d251 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8156746b96cf4f3dbb904e3de3353ae8', 'request_id': 'req-a1386536-471d-4597-b120-88e83eb3d251'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:16:26.175387 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-a1386536-471d-4597-b120-88e83eb3d251 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8156746b96cf4f3dbb904e3de3353ae8', 'request_id': 'req-a1386536-471d-4597-b120-88e83eb3d251'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:16:26.194115 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-a1386536-471d-4597-b120-88e83eb3d251 admin admin] The token cache is being invalidate because project 8156746b96cf4f3dbb904e3de3353ae8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 15:16:26.194587 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-a1386536-471d-4597-b120-88e83eb3d251 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:16:26.195742 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-a1386536-471d-4597-b120-88e83eb3d251 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:16:26.200917 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1248/2483] 10.222.0.46 () {58 vars in 1232 bytes} [Thu May 26 15:16:26 2022] DELETE /identity/v3/projects/8156746b96cf4f3dbb904e3de3353ae8 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:16:26.206325 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0e83af-d2a7-4acd-b8cf-7a6aebea5ef1 service neutron] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:16:26.226290 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0e83af-d2a7-4acd-b8cf-7a6aebea5ef1 service neutron] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:16:26.282274 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:16:26.344662 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfb6df8-5de1-4307-9d85-8160d56df7e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:16:26.345604 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfb6df8-5de1-4307-9d85-8160d56df7e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:16:26.345946 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfb6df8-5de1-4307-9d85-8160d56df7e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:16:26.346232 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfb6df8-5de1-4307-9d85-8160d56df7e3 admin admin] PATH_INFO: `/v3/users/24491c222ac74750a09e3acccaf24bd9` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:16:26.348068 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfb6df8-5de1-4307-9d85-8160d56df7e3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=24491c222ac74750a09e3acccaf24bd9)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:16:26.351604 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfb6df8-5de1-4307-9d85-8160d56df7e3 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:16:26.458962 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-8bfb6df8-5de1-4307-9d85-8160d56df7e3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '24491c222ac74750a09e3acccaf24bd9'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:16:26.479059 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-8bfb6df8-5de1-4307-9d85-8160d56df7e3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '24491c222ac74750a09e3acccaf24bd9'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:16:26.482134 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-8bfb6df8-5de1-4307-9d85-8160d56df7e3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '24491c222ac74750a09e3acccaf24bd9'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:16:26.554536 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-8bfb6df8-5de1-4307-9d85-8160d56df7e3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '24491c222ac74750a09e3acccaf24bd9'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:16:26.567244 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-8bfb6df8-5de1-4307-9d85-8160d56df7e3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '24491c222ac74750a09e3acccaf24bd9'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:16:26.588846 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1236/2484] 10.222.0.46 () {58 vars in 1223 bytes} [Thu May 26 15:16:26 2022] DELETE /identity/v3/users/24491c222ac74750a09e3acccaf24bd9 => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:16:27.020866 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1386536-471d-4597-b120-88e83eb3d251 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:16:27.027491 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-a1386536-471d-4597-b120-88e83eb3d251 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:16:27.052259 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1386536-471d-4597-b120-88e83eb3d251 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:16:27.175610 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0174c07a-5701-4ebb-a53c-7f9ee02138b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:16:27.176832 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0174c07a-5701-4ebb-a53c-7f9ee02138b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:16:27.177157 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0174c07a-5701-4ebb-a53c-7f9ee02138b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:16:27.178293 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0174c07a-5701-4ebb-a53c-7f9ee02138b3 admin admin] PATH_INFO: `/v3/projects/f3d3cb2010644b588a9d699c1ba40d72` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:16:27.179052 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0174c07a-5701-4ebb-a53c-7f9ee02138b3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f3d3cb2010644b588a9d699c1ba40d72)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:16:27.181141 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0174c07a-5701-4ebb-a53c-7f9ee02138b3 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:16:27.287430 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-0174c07a-5701-4ebb-a53c-7f9ee02138b3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f3d3cb2010644b588a9d699c1ba40d72', 'request_id': 'req-0174c07a-5701-4ebb-a53c-7f9ee02138b3'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:16:27.305983 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-0174c07a-5701-4ebb-a53c-7f9ee02138b3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f3d3cb2010644b588a9d699c1ba40d72', 'request_id': 'req-0174c07a-5701-4ebb-a53c-7f9ee02138b3'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:16:27.324826 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-0174c07a-5701-4ebb-a53c-7f9ee02138b3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f3d3cb2010644b588a9d699c1ba40d72', 'request_id': 'req-0174c07a-5701-4ebb-a53c-7f9ee02138b3'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:16:27.350667 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-0174c07a-5701-4ebb-a53c-7f9ee02138b3 admin admin] The token cache is being invalidate because project f3d3cb2010644b588a9d699c1ba40d72 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 15:16:27.351257 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-0174c07a-5701-4ebb-a53c-7f9ee02138b3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:16:27.358483 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-0174c07a-5701-4ebb-a53c-7f9ee02138b3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:16:27.361960 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1249/2485] 10.222.0.46 () {58 vars in 1232 bytes} [Thu May 26 15:16:27 2022] DELETE /identity/v3/projects/f3d3cb2010644b588a9d699c1ba40d72 => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:16:27.370915 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c34ccf0-b1bf-4041-b982-f19c2e9c335d None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:16:27.371561 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c34ccf0-b1bf-4041-b982-f19c2e9c335d None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:16:27.371968 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c34ccf0-b1bf-4041-b982-f19c2e9c335d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:16:27.407828 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-8c34ccf0-b1bf-4041-b982-f19c2e9c335d None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:16:27.410057 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-8c34ccf0-b1bf-4041-b982-f19c2e9c335d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:16:27.535933 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1237/2486] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:16:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 15:16:27.544768 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac56553-e9fa-464e-8ac5-7a0403de8d26 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:16:27.546055 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac56553-e9fa-464e-8ac5-7a0403de8d26 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:16:27.546533 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac56553-e9fa-464e-8ac5-7a0403de8d26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:16:27.565061 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:16:27.574789 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-2ac56553-e9fa-464e-8ac5-7a0403de8d26 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:16:27.576857 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-2ac56553-e9fa-464e-8ac5-7a0403de8d26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:16:27.601300 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1250/2487] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:16:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:16:27.613870 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c34ccf0-b1bf-4041-b982-f19c2e9c335d None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:16:27.640211 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c34ccf0-b1bf-4041-b982-f19c2e9c335d None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:16:27.645801 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95fc539e-07cc-4688-bfa9-c3f57c911099 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:16:27.646810 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fc539e-07cc-4688-bfa9-c3f57c911099 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:16:27.647250 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fc539e-07cc-4688-bfa9-c3f57c911099 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:16:27.647675 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95fc539e-07cc-4688-bfa9-c3f57c911099 admin admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:16:27.648676 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95fc539e-07cc-4688-bfa9-c3f57c911099 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 15:16:27.649144 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95fc539e-07cc-4688-bfa9-c3f57c911099 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:16:27.651061 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95fc539e-07cc-4688-bfa9-c3f57c911099 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:16:27.674625 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1238/2488] 10.222.0.46 () {58 vars in 1152 bytes} [Thu May 26 15:16:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 15:16:27.688369 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac56553-e9fa-464e-8ac5-7a0403de8d26 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:16:27.711981 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac56553-e9fa-464e-8ac5-7a0403de8d26 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:16:27.725803 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e100578f-bd48-4e21-942b-72cdbfe01b8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:16:27.726727 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e100578f-bd48-4e21-942b-72cdbfe01b8b admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:16:27.727012 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e100578f-bd48-4e21-942b-72cdbfe01b8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:16:27.727288 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e100578f-bd48-4e21-942b-72cdbfe01b8b admin admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:16:27.728117 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e100578f-bd48-4e21-942b-72cdbfe01b8b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:16:27.730624 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e100578f-bd48-4e21-942b-72cdbfe01b8b admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:16:27.776315 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3941fe73-7be7-4c16-b911-ed0ce88de145 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:16:27.776817 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3941fe73-7be7-4c16-b911-ed0ce88de145 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:16:27.777321 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3941fe73-7be7-4c16-b911-ed0ce88de145 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:16:27.778723 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1251/2489] 10.222.0.46 () {60 vars in 1152 bytes} [Thu May 26 15:16:27 2022] POST /identity/v3/projects => generated 369 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 15:16:27.786722 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e100578f-bd48-4e21-942b-72cdbfe01b8b admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:16:27.822518 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-3941fe73-7be7-4c16-b911-ed0ce88de145 None None] MFA Rules not processed for user `e6e2622df88f4e1b9b81097d6e558f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:16:27.824578 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-3941fe73-7be7-4c16-b911-ed0ce88de145 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:16:27.874062 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:16:27.918294 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e100578f-bd48-4e21-942b-72cdbfe01b8b admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:16:27.926934 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a656e11-8349-4738-b9a6-4d89eaee949e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:16:27.928020 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a656e11-8349-4738-b9a6-4d89eaee949e admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:16:27.928525 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a656e11-8349-4738-b9a6-4d89eaee949e admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:16:27.929001 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a656e11-8349-4738-b9a6-4d89eaee949e admin admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:16:27.930108 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a656e11-8349-4738-b9a6-4d89eaee949e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:16:27.933406 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a656e11-8349-4738-b9a6-4d89eaee949e admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:16:27.938277 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1239/2490] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:16:27 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 15:16:27.944295 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3941fe73-7be7-4c16-b911-ed0ce88de145 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:16:27.945948 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-3941fe73-7be7-4c16-b911-ed0ce88de145 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:16:27.960792 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3941fe73-7be7-4c16-b911-ed0ce88de145 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:16:27.967993 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1252/2491] 10.222.0.46 () {60 vars in 1143 bytes} [Thu May 26 15:16:27 2022] POST /identity/v3/users => generated 285 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 15:16:27.969408 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c7d440-42e2-4c63-960f-cdbafd4f6ea9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:16:27.970204 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c7d440-42e2-4c63-960f-cdbafd4f6ea9 admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:16:27.970576 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c7d440-42e2-4c63-960f-cdbafd4f6ea9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:16:27.970821 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c7d440-42e2-4c63-960f-cdbafd4f6ea9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:16:27.971599 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c7d440-42e2-4c63-960f-cdbafd4f6ea9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:16:27.974260 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c7d440-42e2-4c63-960f-cdbafd4f6ea9 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:16:27.974503 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a656e11-8349-4738-b9a6-4d89eaee949e admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:16:27.997269 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a656e11-8349-4738-b9a6-4d89eaee949e admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:16:28.003824 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a896d20c-1455-4c7c-a52b-1ce9fb9b4aed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:16:28.004980 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a896d20c-1455-4c7c-a52b-1ce9fb9b4aed admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:16:28.005347 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a896d20c-1455-4c7c-a52b-1ce9fb9b4aed admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:16:28.005917 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a896d20c-1455-4c7c-a52b-1ce9fb9b4aed admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:16:28.006834 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a896d20c-1455-4c7c-a52b-1ce9fb9b4aed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:16:28.008848 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a896d20c-1455-4c7c-a52b-1ce9fb9b4aed admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:16:28.024087 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1240/2492] 10.222.0.46 () {60 vars in 1152 bytes} [Thu May 26 15:16:27 2022] POST /identity/v3/projects => generated 385 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 15:16:28.026595 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1253/2493] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 15:16:27 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 15:16:28.030505 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c7d440-42e2-4c63-960f-cdbafd4f6ea9 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:16:28.031981 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a896d20c-1455-4c7c-a52b-1ce9fb9b4aed admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:16:28.052376 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a896d20c-1455-4c7c-a52b-1ce9fb9b4aed admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:16:28.122735 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c7d440-42e2-4c63-960f-cdbafd4f6ea9 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:16:28.138230 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b02f8d-5635-422c-a9bb-71b091bf8cbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:16:28.140021 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b02f8d-5635-422c-a9bb-71b091bf8cbd admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:16:28.140338 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b02f8d-5635-422c-a9bb-71b091bf8cbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:16:28.140626 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b02f8d-5635-422c-a9bb-71b091bf8cbd admin admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:16:28.141604 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b02f8d-5635-422c-a9bb-71b091bf8cbd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:16:28.143866 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b02f8d-5635-422c-a9bb-71b091bf8cbd admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:16:28.154489 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f9d4cb-fe54-4dc4-aa9c-f58851d44172 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:16:28.155327 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f9d4cb-fe54-4dc4-aa9c-f58851d44172 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:16:28.155629 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f9d4cb-fe54-4dc4-aa9c-f58851d44172 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:16:28.156061 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f9d4cb-fe54-4dc4-aa9c-f58851d44172 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:16:28.156859 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f9d4cb-fe54-4dc4-aa9c-f58851d44172 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:16:28.158664 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f9d4cb-fe54-4dc4-aa9c-f58851d44172 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:16:28.170004 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1241/2494] 10.222.0.46 () {60 vars in 1143 bytes} [Thu May 26 15:16:28 2022] POST /identity/v3/users => generated 299 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 15:16:28.170759 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1254/2495] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 15:16:28 2022] GET /identity/v3/roles => generated 1420 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 15:16:28.176244 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b02f8d-5635-422c-a9bb-71b091bf8cbd admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:16:28.179329 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f9d4cb-fe54-4dc4-aa9c-f58851d44172 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:16:28.189180 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b02f8d-5635-422c-a9bb-71b091bf8cbd admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:16:28.191863 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f9d4cb-fe54-4dc4-aa9c-f58851d44172 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:16:28.195294 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95b0e6c-24d3-4e5c-9012-05fd4948e56d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:16:28.196142 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95b0e6c-24d3-4e5c-9012-05fd4948e56d admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:16:28.196498 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95b0e6c-24d3-4e5c-9012-05fd4948e56d admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:16:28.196855 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95b0e6c-24d3-4e5c-9012-05fd4948e56d admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:16:28.197771 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b36709-5bf5-4b9b-b288-6b7c41f44a8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:16:28.198370 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95b0e6c-24d3-4e5c-9012-05fd4948e56d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:16:28.198869 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b36709-5bf5-4b9b-b288-6b7c41f44a8a admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:16:28.199270 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b36709-5bf5-4b9b-b288-6b7c41f44a8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:16:28.199641 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b36709-5bf5-4b9b-b288-6b7c41f44a8a admin admin] PATH_INFO: `/v3/projects/7ce79c6b4b6a406d90ddf2c59d280646/users/2484f6f26f7a44ddbb5b5369105bf02f/roles/d4088d1845a24fbd9bd8975ce0b1d3d0` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:16:28.200015 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95b0e6c-24d3-4e5c-9012-05fd4948e56d admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:16:28.212306 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1242/2496] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 15:16:28 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 15:16:28.218432 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95b0e6c-24d3-4e5c-9012-05fd4948e56d admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:16:28.224432 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b36709-5bf5-4b9b-b288-6b7c41f44a8a admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ce79c6b4b6a406d90ddf2c59d280646, user_id=2484f6f26f7a44ddbb5b5369105bf02f, role_id=d4088d1845a24fbd9bd8975ce0b1d3d0)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:16:28.227733 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b36709-5bf5-4b9b-b288-6b7c41f44a8a admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:16:28.228876 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95b0e6c-24d3-4e5c-9012-05fd4948e56d admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:16:28.236752 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11f6404-acef-484f-af87-e5ab2d91e80e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:16:28.237850 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11f6404-acef-484f-af87-e5ab2d91e80e admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:16:28.238131 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11f6404-acef-484f-af87-e5ab2d91e80e admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:16:28.238440 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11f6404-acef-484f-af87-e5ab2d91e80e admin admin] PATH_INFO: `/v3/projects/b9d682e03d0d4bfbbf21426fa050fca6/users/eb0306b0e4c74acc849fadf4e8a97367/roles/a2e551773cbd4df084020676865b8222` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:16:28.251717 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1255/2497] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 15:16:28 2022] PUT /identity/v3/projects/7ce79c6b4b6a406d90ddf2c59d280646/users/2484f6f26f7a44ddbb5b5369105bf02f/roles/d4088d1845a24fbd9bd8975ce0b1d3d0 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:16:28.257935 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11f6404-acef-484f-af87-e5ab2d91e80e admin admin] RBAC: Authorizing `identity:create_grant(project_id=b9d682e03d0d4bfbbf21426fa050fca6, user_id=eb0306b0e4c74acc849fadf4e8a97367, role_id=a2e551773cbd4df084020676865b8222)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:16:28.262012 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11f6404-acef-484f-af87-e5ab2d91e80e admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:16:28.269226 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40862b1-4306-4277-a93d-e49c530483f1 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:16:28.269622 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40862b1-4306-4277-a93d-e49c530483f1 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:16:28.270017 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d40862b1-4306-4277-a93d-e49c530483f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:16:28.283716 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1243/2498] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 15:16:28 2022] PUT /identity/v3/projects/b9d682e03d0d4bfbbf21426fa050fca6/users/eb0306b0e4c74acc849fadf4e8a97367/roles/a2e551773cbd4df084020676865b8222 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 15:16:28.288280 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11f6404-acef-484f-af87-e5ab2d91e80e admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:16:28.303872 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-d40862b1-4306-4277-a93d-e49c530483f1 None None] MFA Rules not processed for user `2484f6f26f7a44ddbb5b5369105bf02f`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:16:28.305236 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-d40862b1-4306-4277-a93d-e49c530483f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:16:28.333970 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:16:28.430850 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11f6404-acef-484f-af87-e5ab2d91e80e admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:16:28.440167 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1256/2499] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:16:28 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 15:16:28.443160 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9b8e03-432b-49a6-9e59-15f361230f1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:16:28.443927 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9b8e03-432b-49a6-9e59-15f361230f1f admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:16:28.444202 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9b8e03-432b-49a6-9e59-15f361230f1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:16:28.444602 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9b8e03-432b-49a6-9e59-15f361230f1f admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:16:28.445372 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9b8e03-432b-49a6-9e59-15f361230f1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:16:28.445919 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40862b1-4306-4277-a93d-e49c530483f1 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:16:28.447095 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9b8e03-432b-49a6-9e59-15f361230f1f admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:16:28.450633 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:16:28.463717 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1244/2500] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 15:16:28 2022] GET /identity/v3/roles => generated 1420 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 15:16:28.469608 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d40862b1-4306-4277-a93d-e49c530483f1 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:16:28.474873 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eacd117-9abc-43cf-a512-07cc672355cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:16:28.475270 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9b8e03-432b-49a6-9e59-15f361230f1f admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:16:28.476132 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eacd117-9abc-43cf-a512-07cc672355cb admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:16:28.476484 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eacd117-9abc-43cf-a512-07cc672355cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:16:28.476834 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eacd117-9abc-43cf-a512-07cc672355cb admin admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:16:28.477646 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eacd117-9abc-43cf-a512-07cc672355cb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:16:28.480835 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eacd117-9abc-43cf-a512-07cc672355cb admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:16:28.492428 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9b8e03-432b-49a6-9e59-15f361230f1f admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:16:28.505218 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cbbc2e-213d-43f3-8749-329b939a444d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:16:28.506310 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cbbc2e-213d-43f3-8749-329b939a444d admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:16:28.506660 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cbbc2e-213d-43f3-8749-329b939a444d admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:16:28.507027 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cbbc2e-213d-43f3-8749-329b939a444d admin admin] PATH_INFO: `/v3/projects/b9d682e03d0d4bfbbf21426fa050fca6/users/eb0306b0e4c74acc849fadf4e8a97367/roles/a2e551773cbd4df084020676865b8222` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:16:28.509411 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cbbc2e-213d-43f3-8749-329b939a444d admin admin] RBAC: Authorizing `identity:create_grant(project_id=b9d682e03d0d4bfbbf21426fa050fca6, user_id=eb0306b0e4c74acc849fadf4e8a97367, role_id=a2e551773cbd4df084020676865b8222)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:16:28.512775 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cbbc2e-213d-43f3-8749-329b939a444d admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:16:28.532369 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1245/2501] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 15:16:28 2022] PUT /identity/v3/projects/b9d682e03d0d4bfbbf21426fa050fca6/users/eb0306b0e4c74acc849fadf4e8a97367/roles/a2e551773cbd4df084020676865b8222 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:16:28.539101 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1257/2502] 10.222.0.46 () {60 vars in 1152 bytes} [Thu May 26 15:16:28 2022] POST /identity/v3/projects => generated 369 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 15:16:28.544733 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cbbc2e-213d-43f3-8749-329b939a444d admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:16:28.658858 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cbbc2e-213d-43f3-8749-329b939a444d admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:16:28.768876 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b394c07f-0a17-415f-93ac-0af629a0d2c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:16:28.769786 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b394c07f-0a17-415f-93ac-0af629a0d2c5 admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:16:28.770284 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b394c07f-0a17-415f-93ac-0af629a0d2c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:16:28.770658 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b394c07f-0a17-415f-93ac-0af629a0d2c5 admin admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:16:28.772852 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b394c07f-0a17-415f-93ac-0af629a0d2c5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:16:28.774265 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b394c07f-0a17-415f-93ac-0af629a0d2c5 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:16:28.807595 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1246/2503] 10.222.0.46 () {60 vars in 1143 bytes} [Thu May 26 15:16:28 2022] POST /identity/v3/users => generated 291 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 15:16:28.813262 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eacd117-9abc-43cf-a512-07cc672355cb admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:16:28.830035 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eacd117-9abc-43cf-a512-07cc672355cb admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:16:28.840396 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81664019-dc0a-4f9f-81dd-013b05102b35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:16:28.840567 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81664019-dc0a-4f9f-81dd-013b05102b35 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:16:28.841179 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81664019-dc0a-4f9f-81dd-013b05102b35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:16:28.841179 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81664019-dc0a-4f9f-81dd-013b05102b35 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:16:28.841980 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81664019-dc0a-4f9f-81dd-013b05102b35 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:16:28.843620 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81664019-dc0a-4f9f-81dd-013b05102b35 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:16:28.860056 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1258/2504] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 15:16:28 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 15:16:28.864796 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b394c07f-0a17-415f-93ac-0af629a0d2c5 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:16:28.879516 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b394c07f-0a17-415f-93ac-0af629a0d2c5 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:16:28.885573 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a62336b-3d53-416f-9b91-d5344c5710bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:16:28.886587 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a62336b-3d53-416f-9b91-d5344c5710bd admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:16:28.887504 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a62336b-3d53-416f-9b91-d5344c5710bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:16:28.887504 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a62336b-3d53-416f-9b91-d5344c5710bd admin admin] PATH_INFO: `/v3/projects/6036c969ce5043578c5891e830202e79/users/2498988217b54e71b445a1a5d1ce33d9/roles/a2e551773cbd4df084020676865b8222` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:16:28.919797 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a62336b-3d53-416f-9b91-d5344c5710bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=6036c969ce5043578c5891e830202e79, user_id=2498988217b54e71b445a1a5d1ce33d9, role_id=a2e551773cbd4df084020676865b8222)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:16:28.922783 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a62336b-3d53-416f-9b91-d5344c5710bd admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:16:28.946339 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1247/2505] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 15:16:28 2022] PUT /identity/v3/projects/6036c969ce5043578c5891e830202e79/users/2498988217b54e71b445a1a5d1ce33d9/roles/a2e551773cbd4df084020676865b8222 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:16:28.950876 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81664019-dc0a-4f9f-81dd-013b05102b35 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:16:28.964975 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81664019-dc0a-4f9f-81dd-013b05102b35 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:16:29.071733 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5e1936-11fd-4df8-877c-b8347fc7091f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:16:29.073116 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5e1936-11fd-4df8-877c-b8347fc7091f admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:16:29.073536 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5e1936-11fd-4df8-877c-b8347fc7091f admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:16:29.076130 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5e1936-11fd-4df8-877c-b8347fc7091f admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:16:29.076130 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5e1936-11fd-4df8-877c-b8347fc7091f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:16:29.077584 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5e1936-11fd-4df8-877c-b8347fc7091f admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:16:29.106936 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1259/2506] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 15:16:28 2022] GET /identity/v3/roles => generated 1420 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 15:16:29.108765 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a62336b-3d53-416f-9b91-d5344c5710bd admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:16:29.128675 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a62336b-3d53-416f-9b91-d5344c5710bd admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:16:29.140076 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6dca3c2-f16e-4bc1-b55e-add08a774a0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:16:29.141054 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dca3c2-f16e-4bc1-b55e-add08a774a0f admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:16:29.141347 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dca3c2-f16e-4bc1-b55e-add08a774a0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:16:29.141663 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dca3c2-f16e-4bc1-b55e-add08a774a0f admin admin] PATH_INFO: `/v3/projects/6036c969ce5043578c5891e830202e79/users/2498988217b54e71b445a1a5d1ce33d9/roles/a2e551773cbd4df084020676865b8222` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:16:29.143759 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6dca3c2-f16e-4bc1-b55e-add08a774a0f admin admin] RBAC: Authorizing `identity:create_grant(project_id=6036c969ce5043578c5891e830202e79, user_id=2498988217b54e71b445a1a5d1ce33d9, role_id=a2e551773cbd4df084020676865b8222)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:16:29.146785 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6dca3c2-f16e-4bc1-b55e-add08a774a0f admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:16:29.194011 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1248/2507] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 15:16:29 2022] PUT /identity/v3/projects/6036c969ce5043578c5891e830202e79/users/2498988217b54e71b445a1a5d1ce33d9/roles/a2e551773cbd4df084020676865b8222 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:16:29.205550 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc4ba78-8d61-481e-8c7b-6298e1a80d54 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:16:29.206854 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc4ba78-8d61-481e-8c7b-6298e1a80d54 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:16:29.209367 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc4ba78-8d61-481e-8c7b-6298e1a80d54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:16:29.264448 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-6cc4ba78-8d61-481e-8c7b-6298e1a80d54 None None] MFA Rules not processed for user `2498988217b54e71b445a1a5d1ce33d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:16:29.266796 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-6cc4ba78-8d61-481e-8c7b-6298e1a80d54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:16:29.412364 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1260/2508] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:16:29 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:16:29.430303 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6dca3c2-f16e-4bc1-b55e-add08a774a0f admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:16:29.432742 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-b6dca3c2-f16e-4bc1-b55e-add08a774a0f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:16:29.508291 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:16:29.538313 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6dca3c2-f16e-4bc1-b55e-add08a774a0f admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:16:29.548848 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1a9d4d-4abd-4fdf-b5b8-1a911c1c1ad6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5b5b35614a547d783e6e77ddb8c1c03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:16:29.549844 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1a9d4d-4abd-4fdf-b5b8-1a911c1c1ad6 service cinder] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:16:29.550194 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1a9d4d-4abd-4fdf-b5b8-1a911c1c1ad6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:16:29.550584 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1a9d4d-4abd-4fdf-b5b8-1a911c1c1ad6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:16:29.573334 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1a9d4d-4abd-4fdf-b5b8-1a911c1c1ad6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:16:29.579619 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a1a9d4d-4abd-4fdf-b5b8-1a911c1c1ad6 service cinder] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:16:29.593323 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1249/2509] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:16:29 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 15:16:29.700312 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc4ba78-8d61-481e-8c7b-6298e1a80d54 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:16:29.725984 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc4ba78-8d61-481e-8c7b-6298e1a80d54 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:16:29.736710 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a0e751-38c3-402a-a425-da2957bd4c81 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5b5b35614a547d783e6e77ddb8c1c03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:16:29.738237 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a0e751-38c3-402a-a425-da2957bd4c81 service cinder] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:16:29.738857 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a0e751-38c3-402a-a425-da2957bd4c81 service cinder] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:16:29.739556 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a0e751-38c3-402a-a425-da2957bd4c81 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:16:29.748424 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:16:29.896745 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a0e751-38c3-402a-a425-da2957bd4c81 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:16:29.898687 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a0e751-38c3-402a-a425-da2957bd4c81 service cinder] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:16:29.916937 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1261/2510] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:16:29 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 15:16:33.553073 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1a9d4d-4abd-4fdf-b5b8-1a911c1c1ad6 service cinder] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:16:33.555222 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-2a1a9d4d-4abd-4fdf-b5b8-1a911c1c1ad6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:16:33.801967 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1a9d4d-4abd-4fdf-b5b8-1a911c1c1ad6 service cinder] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:16:33.817027 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8cdac3-8654-44ec-a3d7-d307036c94c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:16:33.818820 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8cdac3-8654-44ec-a3d7-d307036c94c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:16:33.820207 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8cdac3-8654-44ec-a3d7-d307036c94c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:16:33.820207 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8cdac3-8654-44ec-a3d7-d307036c94c1 admin admin] PATH_INFO: `/v3/users/874aa9a1b6cc4f98bbf2e45989792595` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:16:33.823867 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8cdac3-8654-44ec-a3d7-d307036c94c1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=874aa9a1b6cc4f98bbf2e45989792595)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:16:33.826518 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8cdac3-8654-44ec-a3d7-d307036c94c1 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:16:33.936693 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-2b8cdac3-8654-44ec-a3d7-d307036c94c1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '874aa9a1b6cc4f98bbf2e45989792595'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:16:33.953159 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-2b8cdac3-8654-44ec-a3d7-d307036c94c1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '874aa9a1b6cc4f98bbf2e45989792595'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:16:33.954039 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-2b8cdac3-8654-44ec-a3d7-d307036c94c1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '874aa9a1b6cc4f98bbf2e45989792595'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:16:33.977363 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-2b8cdac3-8654-44ec-a3d7-d307036c94c1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '874aa9a1b6cc4f98bbf2e45989792595'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:16:33.993416 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-2b8cdac3-8654-44ec-a3d7-d307036c94c1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '874aa9a1b6cc4f98bbf2e45989792595'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:16:34.001647 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1250/2511] 10.222.0.46 () {58 vars in 1223 bytes} [Thu May 26 15:16:33 2022] DELETE /identity/v3/users/874aa9a1b6cc4f98bbf2e45989792595 => generated 0 bytes in 449 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:16:34.007240 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69048fbf-de15-4600-814d-f9543b068db2 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:16:34.008113 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69048fbf-de15-4600-814d-f9543b068db2 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:16:34.008113 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69048fbf-de15-4600-814d-f9543b068db2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:16:34.029955 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-69048fbf-de15-4600-814d-f9543b068db2 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:16:34.031207 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-69048fbf-de15-4600-814d-f9543b068db2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:16:34.152430 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1262/2512] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:16:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:16:34.163283 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8cdac3-8654-44ec-a3d7-d307036c94c1 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:16:34.165127 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-2b8cdac3-8654-44ec-a3d7-d307036c94c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:16:34.386314 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8cdac3-8654-44ec-a3d7-d307036c94c1 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:16:34.396187 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e54716a-536c-456f-9259-d4b7edb5b3b9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0156a2f6f3794f479e76b85bdbbe5c26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:16:34.397903 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e54716a-536c-456f-9259-d4b7edb5b3b9 service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:16:34.399488 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e54716a-536c-456f-9259-d4b7edb5b3b9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:16:34.400149 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e54716a-536c-456f-9259-d4b7edb5b3b9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:16:34.441343 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e54716a-536c-456f-9259-d4b7edb5b3b9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:16:34.446989 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e54716a-536c-456f-9259-d4b7edb5b3b9 service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:16:34.489882 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1251/2513] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:16:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 15:16:34.975210 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69048fbf-de15-4600-814d-f9543b068db2 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:16:34.977631 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-69048fbf-de15-4600-814d-f9543b068db2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:16:35.020237 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69048fbf-de15-4600-814d-f9543b068db2 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:16:35.044708 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d91592a-dc6b-4684-80b1-895c57eaf106 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:16:35.046195 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d91592a-dc6b-4684-80b1-895c57eaf106 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:16:35.046694 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d91592a-dc6b-4684-80b1-895c57eaf106 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:16:35.047205 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d91592a-dc6b-4684-80b1-895c57eaf106 admin admin] PATH_INFO: `/v3/users/2484f6f26f7a44ddbb5b5369105bf02f` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:16:35.050754 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d91592a-dc6b-4684-80b1-895c57eaf106 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2484f6f26f7a44ddbb5b5369105bf02f)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:16:35.054603 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d91592a-dc6b-4684-80b1-895c57eaf106 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:16:35.125137 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e54716a-536c-456f-9259-d4b7edb5b3b9 service neutron] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:16:35.131571 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-8e54716a-536c-456f-9259-d4b7edb5b3b9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:16:35.157534 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e54716a-536c-456f-9259-d4b7edb5b3b9 service neutron] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:16:35.176592 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002634c6-8211-47aa-9984-84e235be17d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:16:35.178069 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002634c6-8211-47aa-9984-84e235be17d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:16:35.178491 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002634c6-8211-47aa-9984-84e235be17d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:16:35.178780 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002634c6-8211-47aa-9984-84e235be17d3 admin admin] PATH_INFO: `/v3/projects/2a12b93f082045bab1f500827b6c5181` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:16:35.180718 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002634c6-8211-47aa-9984-84e235be17d3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2a12b93f082045bab1f500827b6c5181)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:16:35.188004 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002634c6-8211-47aa-9984-84e235be17d3 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:16:35.193392 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:16:35.232012 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-7d91592a-dc6b-4684-80b1-895c57eaf106 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2484f6f26f7a44ddbb5b5369105bf02f'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:16:35.258301 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-7d91592a-dc6b-4684-80b1-895c57eaf106 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2484f6f26f7a44ddbb5b5369105bf02f'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:16:35.266201 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-7d91592a-dc6b-4684-80b1-895c57eaf106 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2484f6f26f7a44ddbb5b5369105bf02f'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:16:35.314684 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-7d91592a-dc6b-4684-80b1-895c57eaf106 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2484f6f26f7a44ddbb5b5369105bf02f'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:16:35.346857 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-7d91592a-dc6b-4684-80b1-895c57eaf106 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2484f6f26f7a44ddbb5b5369105bf02f'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:16:35.365513 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1263/2514] 10.222.0.46 () {58 vars in 1223 bytes} [Thu May 26 15:16:34 2022] DELETE /identity/v3/users/2484f6f26f7a44ddbb5b5369105bf02f => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:16:35.368262 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-002634c6-8211-47aa-9984-84e235be17d3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2a12b93f082045bab1f500827b6c5181', 'request_id': 'req-002634c6-8211-47aa-9984-84e235be17d3'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:16:35.371729 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b018a7-de4b-40b8-85af-6fca9d4ca4a9 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:16:35.373997 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b018a7-de4b-40b8-85af-6fca9d4ca4a9 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:16:35.374458 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b018a7-de4b-40b8-85af-6fca9d4ca4a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:16:35.390232 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-002634c6-8211-47aa-9984-84e235be17d3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2a12b93f082045bab1f500827b6c5181', 'request_id': 'req-002634c6-8211-47aa-9984-84e235be17d3'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:16:35.407413 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-002634c6-8211-47aa-9984-84e235be17d3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2a12b93f082045bab1f500827b6c5181', 'request_id': 'req-002634c6-8211-47aa-9984-84e235be17d3'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:16:35.419846 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-27b018a7-de4b-40b8-85af-6fca9d4ca4a9 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:16:35.421632 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-27b018a7-de4b-40b8-85af-6fca9d4ca4a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:16:35.439442 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-002634c6-8211-47aa-9984-84e235be17d3 admin admin] The token cache is being invalidate because project 2a12b93f082045bab1f500827b6c5181 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 15:16:35.440169 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-002634c6-8211-47aa-9984-84e235be17d3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:16:35.442150 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-002634c6-8211-47aa-9984-84e235be17d3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:16:35.448180 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1252/2515] 10.222.0.46 () {58 vars in 1232 bytes} [Thu May 26 15:16:35 2022] DELETE /identity/v3/projects/2a12b93f082045bab1f500827b6c5181 => generated 0 bytes in 324 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:16:35.452874 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002634c6-8211-47aa-9984-84e235be17d3 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:16:35.458236 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-002634c6-8211-47aa-9984-84e235be17d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:16:35.476382 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:16:35.584179 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1264/2516] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:16:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:16:35.597743 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b018a7-de4b-40b8-85af-6fca9d4ca4a9 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:16:35.602574 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-27b018a7-de4b-40b8-85af-6fca9d4ca4a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:16:35.699369 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002634c6-8211-47aa-9984-84e235be17d3 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:16:35.714884 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b018a7-de4b-40b8-85af-6fca9d4ca4a9 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:16:35.717786 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4acd58-2ce6-49eb-9795-0e1f90433de1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:16:35.719003 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151abc69-a0d2-43a7-900a-8b3dbb158433 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0156a2f6f3794f479e76b85bdbbe5c26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:16:35.719737 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151abc69-a0d2-43a7-900a-8b3dbb158433 service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:16:35.720031 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151abc69-a0d2-43a7-900a-8b3dbb158433 service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:16:35.720367 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151abc69-a0d2-43a7-900a-8b3dbb158433 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:16:35.725158 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4acd58-2ce6-49eb-9795-0e1f90433de1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:16:35.725823 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4acd58-2ce6-49eb-9795-0e1f90433de1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:16:35.729697 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4acd58-2ce6-49eb-9795-0e1f90433de1 admin admin] PATH_INFO: `/v3/users/4aba3d8abdfb427b9027067b69c0ac42` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:16:35.732636 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4acd58-2ce6-49eb-9795-0e1f90433de1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4aba3d8abdfb427b9027067b69c0ac42)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:16:35.734855 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151abc69-a0d2-43a7-900a-8b3dbb158433 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:16:35.736552 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151abc69-a0d2-43a7-900a-8b3dbb158433 service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:16:35.757707 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4acd58-2ce6-49eb-9795-0e1f90433de1 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:16:35.767113 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1265/2517] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:16:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 15:16:35.934390 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-1c4acd58-2ce6-49eb-9795-0e1f90433de1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4aba3d8abdfb427b9027067b69c0ac42'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:16:35.951738 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-1c4acd58-2ce6-49eb-9795-0e1f90433de1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4aba3d8abdfb427b9027067b69c0ac42'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:16:35.952746 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-1c4acd58-2ce6-49eb-9795-0e1f90433de1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4aba3d8abdfb427b9027067b69c0ac42'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:16:36.013325 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-1c4acd58-2ce6-49eb-9795-0e1f90433de1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4aba3d8abdfb427b9027067b69c0ac42'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:16:36.047272 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-1c4acd58-2ce6-49eb-9795-0e1f90433de1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4aba3d8abdfb427b9027067b69c0ac42'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:16:36.065515 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1253/2518] 10.222.0.46 () {58 vars in 1223 bytes} [Thu May 26 15:16:35 2022] DELETE /identity/v3/users/4aba3d8abdfb427b9027067b69c0ac42 => generated 0 bytes in 613 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:16:36.143630 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a09a2df-02af-4f1a-9a14-cd815f0460a1 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:16:36.144544 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a09a2df-02af-4f1a-9a14-cd815f0460a1 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:16:36.144984 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a09a2df-02af-4f1a-9a14-cd815f0460a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:16:36.190185 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-3a09a2df-02af-4f1a-9a14-cd815f0460a1 None None] MFA Rules not processed for user `eb0306b0e4c74acc849fadf4e8a97367`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:16:36.192829 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-3a09a2df-02af-4f1a-9a14-cd815f0460a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:16:36.288663 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4acd58-2ce6-49eb-9795-0e1f90433de1 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:16:36.291742 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-1c4acd58-2ce6-49eb-9795-0e1f90433de1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:16:36.314839 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:16:36.360475 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1266/2519] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:16:36 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:16:36.364754 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a09a2df-02af-4f1a-9a14-cd815f0460a1 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:16:36.370270 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-3a09a2df-02af-4f1a-9a14-cd815f0460a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:16:36.428494 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4acd58-2ce6-49eb-9795-0e1f90433de1 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:16:36.437733 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c2c3ab-764c-48d2-af7f-c411fb710cbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:16:36.439457 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c2c3ab-764c-48d2-af7f-c411fb710cbd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:16:36.440623 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c2c3ab-764c-48d2-af7f-c411fb710cbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:16:36.441501 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c2c3ab-764c-48d2-af7f-c411fb710cbd admin admin] PATH_INFO: `/v3/projects/7ce79c6b4b6a406d90ddf2c59d280646` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:16:36.443218 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c2c3ab-764c-48d2-af7f-c411fb710cbd admin admin] RBAC: Authorizing `identity:delete_project(project_id=7ce79c6b4b6a406d90ddf2c59d280646)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:16:36.448848 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c2c3ab-764c-48d2-af7f-c411fb710cbd admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:16:36.461066 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:16:36.467044 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a09a2df-02af-4f1a-9a14-cd815f0460a1 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:16:36.471781 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45965a9-844b-4348-b92b-fad49e274ff2 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a436dbdee624f44ad76cca8416b0ad1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:16:36.472452 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45965a9-844b-4348-b92b-fad49e274ff2 service glance] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:16:36.472741 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45965a9-844b-4348-b92b-fad49e274ff2 service glance] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:16:36.473024 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45965a9-844b-4348-b92b-fad49e274ff2 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:16:36.475193 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-d45965a9-844b-4348-b92b-fad49e274ff2 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:16:36.543969 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-30c2c3ab-764c-48d2-af7f-c411fb710cbd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7ce79c6b4b6a406d90ddf2c59d280646', 'request_id': 'req-30c2c3ab-764c-48d2-af7f-c411fb710cbd'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:16:36.562589 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-30c2c3ab-764c-48d2-af7f-c411fb710cbd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7ce79c6b4b6a406d90ddf2c59d280646', 'request_id': 'req-30c2c3ab-764c-48d2-af7f-c411fb710cbd'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:16:36.573257 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-30c2c3ab-764c-48d2-af7f-c411fb710cbd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7ce79c6b4b6a406d90ddf2c59d280646', 'request_id': 'req-30c2c3ab-764c-48d2-af7f-c411fb710cbd'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:16:36.604990 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-30c2c3ab-764c-48d2-af7f-c411fb710cbd admin admin] The token cache is being invalidate because project 7ce79c6b4b6a406d90ddf2c59d280646 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 15:16:36.608749 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-30c2c3ab-764c-48d2-af7f-c411fb710cbd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:16:36.615005 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-30c2c3ab-764c-48d2-af7f-c411fb710cbd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:16:36.624823 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1254/2520] 10.222.0.46 () {58 vars in 1232 bytes} [Thu May 26 15:16:36 2022] DELETE /identity/v3/projects/7ce79c6b4b6a406d90ddf2c59d280646 => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 15:16:36.624823 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c2c3ab-764c-48d2-af7f-c411fb710cbd admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:16:36.625616 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-30c2c3ab-764c-48d2-af7f-c411fb710cbd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:16:36.667773 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45965a9-844b-4348-b92b-fad49e274ff2 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:16:36.667773 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45965a9-844b-4348-b92b-fad49e274ff2 service glance] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:16:36.705790 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1267/2521] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:16:36 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 15:16:36.706916 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45965a9-844b-4348-b92b-fad49e274ff2 service glance] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:16:36.713861 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-d45965a9-844b-4348-b92b-fad49e274ff2 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:16:36.754169 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45965a9-844b-4348-b92b-fad49e274ff2 service glance] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:16:36.868030 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c2c3ab-764c-48d2-af7f-c411fb710cbd admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:16:36.897337 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eee53ab-ed9d-4123-ab6e-a3782214822f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:16:36.898915 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eee53ab-ed9d-4123-ab6e-a3782214822f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:16:36.899418 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eee53ab-ed9d-4123-ab6e-a3782214822f admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:16:36.899964 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eee53ab-ed9d-4123-ab6e-a3782214822f admin admin] PATH_INFO: `/v3/users/2498988217b54e71b445a1a5d1ce33d9` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:16:36.903696 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eee53ab-ed9d-4123-ab6e-a3782214822f admin admin] RBAC: Authorizing `identity:delete_user(user_id=2498988217b54e71b445a1a5d1ce33d9)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:16:36.912081 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eee53ab-ed9d-4123-ab6e-a3782214822f admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:16:36.949022 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f38548b-71e0-4f4c-9392-6cd128151dac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:16:36.950839 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f38548b-71e0-4f4c-9392-6cd128151dac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:16:36.951366 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f38548b-71e0-4f4c-9392-6cd128151dac admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:16:36.960137 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f38548b-71e0-4f4c-9392-6cd128151dac admin admin] PATH_INFO: `/v3/projects/5136e98e54a74f46909ee4c9a8d98c42` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:16:36.965636 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f38548b-71e0-4f4c-9392-6cd128151dac admin admin] RBAC: Authorizing `identity:delete_project(project_id=5136e98e54a74f46909ee4c9a8d98c42)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:16:36.970254 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f38548b-71e0-4f4c-9392-6cd128151dac admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:16:37.067752 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:16:37.095518 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-1eee53ab-ed9d-4123-ab6e-a3782214822f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2498988217b54e71b445a1a5d1ce33d9'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:16:37.114550 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-6f38548b-71e0-4f4c-9392-6cd128151dac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5136e98e54a74f46909ee4c9a8d98c42', 'request_id': 'req-6f38548b-71e0-4f4c-9392-6cd128151dac'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:16:37.114937 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-1eee53ab-ed9d-4123-ab6e-a3782214822f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2498988217b54e71b445a1a5d1ce33d9'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:16:37.115483 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-1eee53ab-ed9d-4123-ab6e-a3782214822f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2498988217b54e71b445a1a5d1ce33d9'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:16:37.131021 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-6f38548b-71e0-4f4c-9392-6cd128151dac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5136e98e54a74f46909ee4c9a8d98c42', 'request_id': 'req-6f38548b-71e0-4f4c-9392-6cd128151dac'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:16:37.154901 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-6f38548b-71e0-4f4c-9392-6cd128151dac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5136e98e54a74f46909ee4c9a8d98c42', 'request_id': 'req-6f38548b-71e0-4f4c-9392-6cd128151dac'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:16:37.166189 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-1eee53ab-ed9d-4123-ab6e-a3782214822f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2498988217b54e71b445a1a5d1ce33d9'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:16:37.169526 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-6f38548b-71e0-4f4c-9392-6cd128151dac admin admin] The token cache is being invalidate because project 5136e98e54a74f46909ee4c9a8d98c42 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 15:16:37.170142 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-6f38548b-71e0-4f4c-9392-6cd128151dac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:16:37.171233 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-6f38548b-71e0-4f4c-9392-6cd128151dac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:16:37.173011 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1268/2522] 10.222.0.46 () {58 vars in 1232 bytes} [Thu May 26 15:16:36 2022] DELETE /identity/v3/projects/5136e98e54a74f46909ee4c9a8d98c42 => generated 0 bytes in 467 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:16:37.180050 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc434334-bf36-4007-89f7-b188edc25f08 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:16:37.180517 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc434334-bf36-4007-89f7-b188edc25f08 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:16:37.180774 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-1eee53ab-ed9d-4123-ab6e-a3782214822f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2498988217b54e71b445a1a5d1ce33d9'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:16:37.181534 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc434334-bf36-4007-89f7-b188edc25f08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:16:37.192260 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1255/2523] 10.222.0.46 () {58 vars in 1223 bytes} [Thu May 26 15:16:36 2022] DELETE /identity/v3/users/2498988217b54e71b445a1a5d1ce33d9 => generated 0 bytes in 569 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:16:37.194239 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e84c6f-55e9-4b4a-9fcb-dc97835a4373 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:16:37.194690 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e84c6f-55e9-4b4a-9fcb-dc97835a4373 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:16:37.195134 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e84c6f-55e9-4b4a-9fcb-dc97835a4373 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:16:37.209244 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-fc434334-bf36-4007-89f7-b188edc25f08 None None] MFA Rules not processed for user `8028c520d93449a6a087c281280ee13d`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:16:37.210814 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-fc434334-bf36-4007-89f7-b188edc25f08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:16:37.231044 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-c0e84c6f-55e9-4b4a-9fcb-dc97835a4373 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:16:37.233292 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-c0e84c6f-55e9-4b4a-9fcb-dc97835a4373 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:16:37.263749 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:16:37.343375 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1269/2524] 10.222.0.46 () {58 vars in 1009 bytes} [Thu May 26 15:16:37 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:16:37.384693 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1256/2525] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:16:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:16:37.396214 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c63f6c-9380-40e6-9631-f39a75fc1d5c None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:16:37.396743 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c63f6c-9380-40e6-9631-f39a75fc1d5c None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:16:37.397680 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c63f6c-9380-40e6-9631-f39a75fc1d5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:16:37.431737 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e84c6f-55e9-4b4a-9fcb-dc97835a4373 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:16:37.433246 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-c0e84c6f-55e9-4b4a-9fcb-dc97835a4373 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:16:37.439067 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-c4c63f6c-9380-40e6-9631-f39a75fc1d5c None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:16:37.446552 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-c4c63f6c-9380-40e6-9631-f39a75fc1d5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:16:37.465355 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1270/2526] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:16:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:16:37.469484 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c63f6c-9380-40e6-9631-f39a75fc1d5c None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:16:37.507292 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c63f6c-9380-40e6-9631-f39a75fc1d5c None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:16:37.514538 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e6e3eb-f03b-4000-8cea-dd6c21bb4b79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:16:37.515283 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e6e3eb-f03b-4000-8cea-dd6c21bb4b79 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:16:37.515741 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e6e3eb-f03b-4000-8cea-dd6c21bb4b79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:16:37.515741 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e6e3eb-f03b-4000-8cea-dd6c21bb4b79 admin admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:16:37.517889 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e6e3eb-f03b-4000-8cea-dd6c21bb4b79 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 15:16:37.518387 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e6e3eb-f03b-4000-8cea-dd6c21bb4b79 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:16:37.520499 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e6e3eb-f03b-4000-8cea-dd6c21bb4b79 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:16:37.549396 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1271/2527] 10.222.0.46 () {58 vars in 1152 bytes} [Thu May 26 15:16:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 15:16:37.551242 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e6e3eb-f03b-4000-8cea-dd6c21bb4b79 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:16:37.582327 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e6e3eb-f03b-4000-8cea-dd6c21bb4b79 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:16:37.593158 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e84c6f-55e9-4b4a-9fcb-dc97835a4373 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:16:37.597928 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609c0413-9e81-4aaa-b5a3-4ac93e7b32f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:16:37.598738 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609c0413-9e81-4aaa-b5a3-4ac93e7b32f0 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:16:37.599039 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609c0413-9e81-4aaa-b5a3-4ac93e7b32f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:16:37.599315 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609c0413-9e81-4aaa-b5a3-4ac93e7b32f0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:16:37.600248 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609c0413-9e81-4aaa-b5a3-4ac93e7b32f0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:16:37.602241 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609c0413-9e81-4aaa-b5a3-4ac93e7b32f0 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:16:37.603219 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4103052-2ff4-43f2-bcb0-a44f98b4a400 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4bea272624b45d8ac2ad27bd07df885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:16:37.604322 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4103052-2ff4-43f2-bcb0-a44f98b4a400 service swift] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:16:37.604818 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4103052-2ff4-43f2-bcb0-a44f98b4a400 service swift] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:16:37.609964 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4103052-2ff4-43f2-bcb0-a44f98b4a400 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:16:37.628557 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4103052-2ff4-43f2-bcb0-a44f98b4a400 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:16:37.631189 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4103052-2ff4-43f2-bcb0-a44f98b4a400 service swift] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:16:37.666589 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1272/2528] 10.222.0.46 () {60 vars in 1152 bytes} [Thu May 26 15:16:37 2022] POST /identity/v3/projects => generated 389 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 15:16:37.666771 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1257/2529] 10.222.0.46 () {60 vars in 1384 bytes} [Thu May 26 15:16:37 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 26 15:16:37.668081 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609c0413-9e81-4aaa-b5a3-4ac93e7b32f0 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:16:37.700289 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4103052-2ff4-43f2-bcb0-a44f98b4a400 service swift] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:16:37.702244 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-f4103052-2ff4-43f2-bcb0-a44f98b4a400 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:16:37.734413 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4103052-2ff4-43f2-bcb0-a44f98b4a400 service swift] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:16:37.824970 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:16:37.846887 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a436955b-a9c8-4dc6-83b2-016dfe94e44e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:16:37.848085 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a436955b-a9c8-4dc6-83b2-016dfe94e44e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:16:37.848867 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a436955b-a9c8-4dc6-83b2-016dfe94e44e admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:16:37.849323 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a436955b-a9c8-4dc6-83b2-016dfe94e44e admin admin] PATH_INFO: `/v3/projects/6036c969ce5043578c5891e830202e79` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:16:37.851518 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a436955b-a9c8-4dc6-83b2-016dfe94e44e admin admin] RBAC: Authorizing `identity:delete_project(project_id=6036c969ce5043578c5891e830202e79)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:16:37.856417 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a436955b-a9c8-4dc6-83b2-016dfe94e44e admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:16:37.858369 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609c0413-9e81-4aaa-b5a3-4ac93e7b32f0 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:16:37.865961 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb9cdb6-c0fe-4229-90b5-4c9ea5606c9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:16:37.866919 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb9cdb6-c0fe-4229-90b5-4c9ea5606c9e admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:16:37.867205 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb9cdb6-c0fe-4229-90b5-4c9ea5606c9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:16:37.867483 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb9cdb6-c0fe-4229-90b5-4c9ea5606c9e admin admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:16:37.871617 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb9cdb6-c0fe-4229-90b5-4c9ea5606c9e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:16:37.875559 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb9cdb6-c0fe-4229-90b5-4c9ea5606c9e admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:16:37.949128 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1273/2530] 10.222.0.46 () {60 vars in 1143 bytes} [Thu May 26 15:16:37 2022] POST /identity/v3/users => generated 295 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 15:16:37.954046 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb9cdb6-c0fe-4229-90b5-4c9ea5606c9e admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:16:37.972740 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb9cdb6-c0fe-4229-90b5-4c9ea5606c9e admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:16:37.999672 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-a436955b-a9c8-4dc6-83b2-016dfe94e44e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6036c969ce5043578c5891e830202e79', 'request_id': 'req-a436955b-a9c8-4dc6-83b2-016dfe94e44e'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:16:38.014023 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-a436955b-a9c8-4dc6-83b2-016dfe94e44e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6036c969ce5043578c5891e830202e79', 'request_id': 'req-a436955b-a9c8-4dc6-83b2-016dfe94e44e'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:16:38.025888 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-a436955b-a9c8-4dc6-83b2-016dfe94e44e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6036c969ce5043578c5891e830202e79', 'request_id': 'req-a436955b-a9c8-4dc6-83b2-016dfe94e44e'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:16:38.055402 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-a436955b-a9c8-4dc6-83b2-016dfe94e44e admin admin] The token cache is being invalidate because project 6036c969ce5043578c5891e830202e79 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 15:16:38.055787 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-a436955b-a9c8-4dc6-83b2-016dfe94e44e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:16:38.057039 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-a436955b-a9c8-4dc6-83b2-016dfe94e44e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:16:38.060284 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1258/2531] 10.222.0.46 () {58 vars in 1232 bytes} [Thu May 26 15:16:37 2022] DELETE /identity/v3/projects/6036c969ce5043578c5891e830202e79 => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:16:38.068100 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506bcd92-dd3e-4d36-b8c2-58f250718a49 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:16:38.068592 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506bcd92-dd3e-4d36-b8c2-58f250718a49 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:16:38.069107 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506bcd92-dd3e-4d36-b8c2-58f250718a49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:16:38.100424 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-506bcd92-dd3e-4d36-b8c2-58f250718a49 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:16:38.101523 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-506bcd92-dd3e-4d36-b8c2-58f250718a49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:16:38.132593 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596d2f1c-b25c-42bc-8630-487f238553ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:16:38.133252 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596d2f1c-b25c-42bc-8630-487f238553ca admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:16:38.133596 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596d2f1c-b25c-42bc-8630-487f238553ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:16:38.133974 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596d2f1c-b25c-42bc-8630-487f238553ca admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:16:38.134897 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596d2f1c-b25c-42bc-8630-487f238553ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:16:38.136572 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596d2f1c-b25c-42bc-8630-487f238553ca admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:16:38.153502 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1274/2532] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 15:16:37 2022] GET /identity/v3/roles => generated 1420 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 15:16:38.159030 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596d2f1c-b25c-42bc-8630-487f238553ca admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:16:38.161417 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-596d2f1c-b25c-42bc-8630-487f238553ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:16:38.198425 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596d2f1c-b25c-42bc-8630-487f238553ca admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:16:38.212476 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d008d05d-d879-44e4-b982-6d48fe2a6578 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:16:38.214469 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d008d05d-d879-44e4-b982-6d48fe2a6578 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:16:38.215430 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d008d05d-d879-44e4-b982-6d48fe2a6578 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:16:38.215931 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d008d05d-d879-44e4-b982-6d48fe2a6578 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:16:38.217678 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d008d05d-d879-44e4-b982-6d48fe2a6578 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:16:38.222513 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d008d05d-d879-44e4-b982-6d48fe2a6578 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:16:38.258263 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1275/2533] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 15:16:38 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 15:16:38.263432 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d008d05d-d879-44e4-b982-6d48fe2a6578 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:16:38.276152 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1259/2534] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:16:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:16:38.284488 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d008d05d-d879-44e4-b982-6d48fe2a6578 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:16:38.285905 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd23a47-9b55-439a-be1a-f32a69f774ee None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:16:38.286269 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd23a47-9b55-439a-be1a-f32a69f774ee None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:16:38.286664 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd23a47-9b55-439a-be1a-f32a69f774ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:16:38.291334 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3755bc68-9eb9-4c32-b150-23a95c262dba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:16:38.292339 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3755bc68-9eb9-4c32-b150-23a95c262dba admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:16:38.292737 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3755bc68-9eb9-4c32-b150-23a95c262dba admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:16:38.293137 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3755bc68-9eb9-4c32-b150-23a95c262dba admin admin] PATH_INFO: `/v3/projects/abd94b4320c746d7a97dd173addfdd8b/users/d252ecc41fa343098a855a55323c6bf7/roles/d4088d1845a24fbd9bd8975ce0b1d3d0` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:16:38.321968 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:16:38.336130 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-4cd23a47-9b55-439a-be1a-f32a69f774ee None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:16:38.340406 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-4cd23a47-9b55-439a-be1a-f32a69f774ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:16:38.355863 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3755bc68-9eb9-4c32-b150-23a95c262dba admin admin] RBAC: Authorizing `identity:create_grant(project_id=abd94b4320c746d7a97dd173addfdd8b, user_id=d252ecc41fa343098a855a55323c6bf7, role_id=d4088d1845a24fbd9bd8975ce0b1d3d0)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:16:38.357389 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1260/2535] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:16:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:16:38.361732 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3755bc68-9eb9-4c32-b150-23a95c262dba admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:16:38.364436 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd23a47-9b55-439a-be1a-f32a69f774ee None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:16:38.382062 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd23a47-9b55-439a-be1a-f32a69f774ee None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:16:38.385723 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1276/2536] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 15:16:38 2022] PUT /identity/v3/projects/abd94b4320c746d7a97dd173addfdd8b/users/d252ecc41fa343098a855a55323c6bf7/roles/d4088d1845a24fbd9bd8975ce0b1d3d0 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 15:16:38.400276 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c742b316-d48a-42cc-abd7-6ffad36b912b None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:16:38.400829 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c742b316-d48a-42cc-abd7-6ffad36b912b None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:16:38.400829 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c742b316-d48a-42cc-abd7-6ffad36b912b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:16:38.440296 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-c742b316-d48a-42cc-abd7-6ffad36b912b None None] MFA Rules not processed for user `d252ecc41fa343098a855a55323c6bf7`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:16:38.450898 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-c742b316-d48a-42cc-abd7-6ffad36b912b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:16:38.528410 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7591f4-cbde-40eb-a75c-6e24dd0765f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:16:38.529208 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7591f4-cbde-40eb-a75c-6e24dd0765f8 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:16:38.529616 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7591f4-cbde-40eb-a75c-6e24dd0765f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:16:38.529969 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7591f4-cbde-40eb-a75c-6e24dd0765f8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:16:38.530841 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7591f4-cbde-40eb-a75c-6e24dd0765f8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 15:16:38.531223 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7591f4-cbde-40eb-a75c-6e24dd0765f8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:16:38.532960 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7591f4-cbde-40eb-a75c-6e24dd0765f8 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:16:38.547681 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1261/2537] 10.222.0.46 () {58 vars in 1152 bytes} [Thu May 26 15:16:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 15:16:38.552324 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7591f4-cbde-40eb-a75c-6e24dd0765f8 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:16:38.566842 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7591f4-cbde-40eb-a75c-6e24dd0765f8 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:16:38.576602 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040c6baa-0776-4775-8391-fb7b4b4c1941 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:16:38.577693 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040c6baa-0776-4775-8391-fb7b4b4c1941 admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:16:38.578283 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040c6baa-0776-4775-8391-fb7b4b4c1941 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:16:38.578490 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040c6baa-0776-4775-8391-fb7b4b4c1941 admin admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:16:38.579506 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040c6baa-0776-4775-8391-fb7b4b4c1941 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:16:38.581922 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1277/2538] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:16:38 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:16:38.585355 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040c6baa-0776-4775-8391-fb7b4b4c1941 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:16:38.586189 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c742b316-d48a-42cc-abd7-6ffad36b912b None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:16:38.590320 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:16:38.605479 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c742b316-d48a-42cc-abd7-6ffad36b912b None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:16:38.639065 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094b1c72-cd50-4087-8f75-f35d469ce43e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:16:38.640432 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094b1c72-cd50-4087-8f75-f35d469ce43e admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:16:38.640760 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094b1c72-cd50-4087-8f75-f35d469ce43e admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:16:38.641127 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094b1c72-cd50-4087-8f75-f35d469ce43e admin admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:16:38.641875 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094b1c72-cd50-4087-8f75-f35d469ce43e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:16:38.644012 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094b1c72-cd50-4087-8f75-f35d469ce43e admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:16:38.654810 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1262/2539] 10.222.0.46 () {60 vars in 1152 bytes} [Thu May 26 15:16:38 2022] POST /identity/v3/projects => generated 407 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 15:16:38.660696 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040c6baa-0776-4775-8391-fb7b4b4c1941 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:16:38.689672 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1278/2540] 10.222.0.46 () {60 vars in 1152 bytes} [Thu May 26 15:16:38 2022] POST /identity/v3/projects => generated 389 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 15:16:38.692800 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094b1c72-cd50-4087-8f75-f35d469ce43e admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:16:38.792429 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040c6baa-0776-4775-8391-fb7b4b4c1941 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:16:38.848219 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094b1c72-cd50-4087-8f75-f35d469ce43e admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:16:38.865024 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de92c11b-c1e0-41e2-a0a5-68f0586b300b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:16:38.865842 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de92c11b-c1e0-41e2-a0a5-68f0586b300b admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:16:38.870045 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de92c11b-c1e0-41e2-a0a5-68f0586b300b admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:16:38.870594 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de92c11b-c1e0-41e2-a0a5-68f0586b300b admin admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:16:38.874540 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de92c11b-c1e0-41e2-a0a5-68f0586b300b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:16:38.876676 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de92c11b-c1e0-41e2-a0a5-68f0586b300b admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:16:38.899319 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4081b46b-387a-4bd4-b861-5c21f1cc6b4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:16:38.900196 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4081b46b-387a-4bd4-b861-5c21f1cc6b4f admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:16:38.900484 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4081b46b-387a-4bd4-b861-5c21f1cc6b4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:16:38.900826 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4081b46b-387a-4bd4-b861-5c21f1cc6b4f admin admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:16:38.901605 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4081b46b-387a-4bd4-b861-5c21f1cc6b4f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:16:38.905732 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4081b46b-387a-4bd4-b861-5c21f1cc6b4f admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:16:38.912672 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1279/2541] 10.222.0.46 () {60 vars in 1143 bytes} [Thu May 26 15:16:38 2022] POST /identity/v3/users => generated 301 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 15:16:38.917496 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de92c11b-c1e0-41e2-a0a5-68f0586b300b admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:16:38.934481 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1263/2542] 10.222.0.46 () {60 vars in 1143 bytes} [Thu May 26 15:16:38 2022] POST /identity/v3/users => generated 304 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 15:16:38.938935 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4081b46b-387a-4bd4-b861-5c21f1cc6b4f admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:16:38.939220 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de92c11b-c1e0-41e2-a0a5-68f0586b300b admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:16:38.951282 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22233f5-59b3-4817-a841-e0b8d484d3cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:16:38.952444 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4081b46b-387a-4bd4-b861-5c21f1cc6b4f admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:16:38.953193 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22233f5-59b3-4817-a841-e0b8d484d3cf admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:16:38.955091 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22233f5-59b3-4817-a841-e0b8d484d3cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:16:38.955477 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22233f5-59b3-4817-a841-e0b8d484d3cf admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:16:38.956328 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22233f5-59b3-4817-a841-e0b8d484d3cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:16:38.958376 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22233f5-59b3-4817-a841-e0b8d484d3cf admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:16:38.962043 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73551ce0-64b9-4ce4-aae7-1b31bf206a40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:16:38.962822 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73551ce0-64b9-4ce4-aae7-1b31bf206a40 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:16:38.963116 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73551ce0-64b9-4ce4-aae7-1b31bf206a40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:16:38.963399 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73551ce0-64b9-4ce4-aae7-1b31bf206a40 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:16:38.964152 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73551ce0-64b9-4ce4-aae7-1b31bf206a40 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:16:38.966136 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73551ce0-64b9-4ce4-aae7-1b31bf206a40 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:16:38.978608 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1280/2543] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 15:16:38 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 15:16:38.982779 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22233f5-59b3-4817-a841-e0b8d484d3cf admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:16:38.988929 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1264/2544] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 15:16:38 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 15:16:38.990128 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73551ce0-64b9-4ce4-aae7-1b31bf206a40 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:16:39.002696 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73551ce0-64b9-4ce4-aae7-1b31bf206a40 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:16:39.011081 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53b5395-2366-4052-8519-e185437b3cbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:16:39.011757 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53b5395-2366-4052-8519-e185437b3cbf admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:16:39.012045 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53b5395-2366-4052-8519-e185437b3cbf admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:16:39.012329 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53b5395-2366-4052-8519-e185437b3cbf admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:16:39.013066 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53b5395-2366-4052-8519-e185437b3cbf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:16:39.016515 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53b5395-2366-4052-8519-e185437b3cbf admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:16:39.026223 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22233f5-59b3-4817-a841-e0b8d484d3cf admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:16:39.033700 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e192490-7715-4003-b1a0-d5295fad2043 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:16:39.035106 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e192490-7715-4003-b1a0-d5295fad2043 admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:16:39.035656 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e192490-7715-4003-b1a0-d5295fad2043 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:16:39.036156 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e192490-7715-4003-b1a0-d5295fad2043 admin admin] PATH_INFO: `/v3/projects/5931da338ccd433c988ebe5f6ccdc076/users/ed4151ab4624462395a3337ccc5e8e3e/roles/a2e551773cbd4df084020676865b8222` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:16:39.045381 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1265/2545] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 15:16:38 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 15:16:39.050210 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53b5395-2366-4052-8519-e185437b3cbf admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:16:39.064764 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53b5395-2366-4052-8519-e185437b3cbf admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:16:39.065106 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e192490-7715-4003-b1a0-d5295fad2043 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5931da338ccd433c988ebe5f6ccdc076, user_id=ed4151ab4624462395a3337ccc5e8e3e, role_id=a2e551773cbd4df084020676865b8222)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:16:39.068004 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e192490-7715-4003-b1a0-d5295fad2043 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:16:39.073140 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac324936-3a76-4cd1-9995-a94daf588648 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:16:39.074364 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac324936-3a76-4cd1-9995-a94daf588648 admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:16:39.074681 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac324936-3a76-4cd1-9995-a94daf588648 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:16:39.074973 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac324936-3a76-4cd1-9995-a94daf588648 admin admin] PATH_INFO: `/v3/projects/0d40a0d3f8704d6bbc0f761581270ad7/users/aab211eb3c664ece82f090ce0cfc6658/roles/d4088d1845a24fbd9bd8975ce0b1d3d0` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:16:39.096155 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac324936-3a76-4cd1-9995-a94daf588648 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0d40a0d3f8704d6bbc0f761581270ad7, user_id=aab211eb3c664ece82f090ce0cfc6658, role_id=d4088d1845a24fbd9bd8975ce0b1d3d0)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:16:39.099400 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac324936-3a76-4cd1-9995-a94daf588648 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:16:39.109688 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1281/2546] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 15:16:38 2022] PUT /identity/v3/projects/5931da338ccd433c988ebe5f6ccdc076/users/ed4151ab4624462395a3337ccc5e8e3e/roles/a2e551773cbd4df084020676865b8222 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:16:39.114746 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e192490-7715-4003-b1a0-d5295fad2043 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:16:39.131919 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:16:39.149658 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1266/2547] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 15:16:39 2022] PUT /identity/v3/projects/0d40a0d3f8704d6bbc0f761581270ad7/users/aab211eb3c664ece82f090ce0cfc6658/roles/d4088d1845a24fbd9bd8975ce0b1d3d0 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:16:39.183187 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27b25d9-28c9-4bac-877c-6773d226f5e8 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:16:39.183630 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27b25d9-28c9-4bac-877c-6773d226f5e8 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:16:39.183798 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27b25d9-28c9-4bac-877c-6773d226f5e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:16:39.252797 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-f27b25d9-28c9-4bac-877c-6773d226f5e8 None None] MFA Rules not processed for user `aab211eb3c664ece82f090ce0cfc6658`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:16:39.255369 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-f27b25d9-28c9-4bac-877c-6773d226f5e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:16:39.282616 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e192490-7715-4003-b1a0-d5295fad2043 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:16:39.314003 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:16:39.430660 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1267/2548] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:16:39 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:16:39.434311 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27b25d9-28c9-4bac-877c-6773d226f5e8 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:16:39.438618 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b2a9f3-a6c7-44c8-ab17-6862eaa2e864 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:16:39.439887 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b2a9f3-a6c7-44c8-ab17-6862eaa2e864 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:16:39.440185 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b2a9f3-a6c7-44c8-ab17-6862eaa2e864 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:16:39.440474 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b2a9f3-a6c7-44c8-ab17-6862eaa2e864 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:16:39.441286 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b2a9f3-a6c7-44c8-ab17-6862eaa2e864 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:16:39.443460 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b2a9f3-a6c7-44c8-ab17-6862eaa2e864 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:16:39.468021 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27b25d9-28c9-4bac-877c-6773d226f5e8 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:16:39.471649 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1282/2549] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 15:16:39 2022] GET /identity/v3/roles => generated 1420 bytes in 358 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 15:16:39.474988 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b2a9f3-a6c7-44c8-ab17-6862eaa2e864 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:16:39.483121 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff2a43c-d1d2-4341-9253-f28faaf3f04e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:16:39.485579 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff2a43c-d1d2-4341-9253-f28faaf3f04e admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:16:39.486035 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff2a43c-d1d2-4341-9253-f28faaf3f04e admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:16:39.486573 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff2a43c-d1d2-4341-9253-f28faaf3f04e admin admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:16:39.488937 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff2a43c-d1d2-4341-9253-f28faaf3f04e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:16:39.491106 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff2a43c-d1d2-4341-9253-f28faaf3f04e admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:16:39.503613 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b2a9f3-a6c7-44c8-ab17-6862eaa2e864 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:16:39.523766 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a25b83-59a9-4a3c-b826-92a1a8a2659c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:16:39.525222 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a25b83-59a9-4a3c-b826-92a1a8a2659c admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:16:39.525551 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a25b83-59a9-4a3c-b826-92a1a8a2659c admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:16:39.526319 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a25b83-59a9-4a3c-b826-92a1a8a2659c admin admin] PATH_INFO: `/v3/projects/5931da338ccd433c988ebe5f6ccdc076/users/ed4151ab4624462395a3337ccc5e8e3e/roles/a2e551773cbd4df084020676865b8222` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:16:39.530103 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a25b83-59a9-4a3c-b826-92a1a8a2659c admin admin] RBAC: Authorizing `identity:create_grant(project_id=5931da338ccd433c988ebe5f6ccdc076, user_id=ed4151ab4624462395a3337ccc5e8e3e, role_id=a2e551773cbd4df084020676865b8222)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:16:39.533656 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a25b83-59a9-4a3c-b826-92a1a8a2659c admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:16:39.569518 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1283/2550] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 15:16:39 2022] PUT /identity/v3/projects/5931da338ccd433c988ebe5f6ccdc076/users/ed4151ab4624462395a3337ccc5e8e3e/roles/a2e551773cbd4df084020676865b8222 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:16:39.578119 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fca2dc9-01b1-40d3-8c51-26f9c3e05931 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:16:39.578594 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fca2dc9-01b1-40d3-8c51-26f9c3e05931 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:16:39.578980 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fca2dc9-01b1-40d3-8c51-26f9c3e05931 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:16:39.582304 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1268/2551] 10.222.0.46 () {60 vars in 1152 bytes} [Thu May 26 15:16:39 2022] POST /identity/v3/projects => generated 409 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 15:16:39.588115 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff2a43c-d1d2-4341-9253-f28faaf3f04e admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:16:39.648486 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-3fca2dc9-01b1-40d3-8c51-26f9c3e05931 None None] MFA Rules not processed for user `ed4151ab4624462395a3337ccc5e8e3e`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:16:39.651321 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-3fca2dc9-01b1-40d3-8c51-26f9c3e05931 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:16:39.792838 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff2a43c-d1d2-4341-9253-f28faaf3f04e admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:16:39.805030 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce13b82-4d74-45b5-a397-28f8682fec21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:16:39.806935 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce13b82-4d74-45b5-a397-28f8682fec21 admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:16:39.807831 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce13b82-4d74-45b5-a397-28f8682fec21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:16:39.808593 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce13b82-4d74-45b5-a397-28f8682fec21 admin admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:16:39.810241 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce13b82-4d74-45b5-a397-28f8682fec21 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:16:39.813033 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce13b82-4d74-45b5-a397-28f8682fec21 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:16:39.822715 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1284/2552] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:16:39 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:16:39.840711 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fca2dc9-01b1-40d3-8c51-26f9c3e05931 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:16:39.843251 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-3fca2dc9-01b1-40d3-8c51-26f9c3e05931 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:16:39.860469 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1269/2553] 10.222.0.46 () {60 vars in 1143 bytes} [Thu May 26 15:16:39 2022] POST /identity/v3/users => generated 311 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 15:16:39.866848 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce13b82-4d74-45b5-a397-28f8682fec21 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:16:39.882986 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce13b82-4d74-45b5-a397-28f8682fec21 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:16:39.889535 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1211e621-da29-439c-aed8-dcd0a47ca7c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:16:39.890230 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1211e621-da29-439c-aed8-dcd0a47ca7c0 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:16:39.890548 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1211e621-da29-439c-aed8-dcd0a47ca7c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:16:39.890841 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1211e621-da29-439c-aed8-dcd0a47ca7c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:16:39.891958 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1211e621-da29-439c-aed8-dcd0a47ca7c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:16:39.893666 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1211e621-da29-439c-aed8-dcd0a47ca7c0 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:16:39.901915 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:16:39.918872 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1270/2554] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 15:16:39 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 15:16:39.922066 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1211e621-da29-439c-aed8-dcd0a47ca7c0 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:16:39.953142 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1211e621-da29-439c-aed8-dcd0a47ca7c0 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:16:39.966580 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b71e290-8b16-49ec-8d89-50a1eb9105e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:16:39.967488 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b71e290-8b16-49ec-8d89-50a1eb9105e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:16:39.967768 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b71e290-8b16-49ec-8d89-50a1eb9105e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:16:39.968045 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b71e290-8b16-49ec-8d89-50a1eb9105e8 admin admin] PATH_INFO: `/v3/projects/fef4bb5baec149eeae8fa64e52ee0f97/users/b32c02ae04e5439aabf9d0f78c3868c9/roles/a2e551773cbd4df084020676865b8222` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:16:39.980391 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fca2dc9-01b1-40d3-8c51-26f9c3e05931 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:16:39.986471 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a565db12-9004-4580-8743-d3e672ac200c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5b5b35614a547d783e6e77ddb8c1c03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:16:39.987228 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a565db12-9004-4580-8743-d3e672ac200c service cinder] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:16:39.987449 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a565db12-9004-4580-8743-d3e672ac200c service cinder] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:16:39.987730 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a565db12-9004-4580-8743-d3e672ac200c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:16:39.999917 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b71e290-8b16-49ec-8d89-50a1eb9105e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fef4bb5baec149eeae8fa64e52ee0f97, user_id=b32c02ae04e5439aabf9d0f78c3868c9, role_id=a2e551773cbd4df084020676865b8222)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:16:40.008131 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a565db12-9004-4580-8743-d3e672ac200c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:16:40.010316 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b71e290-8b16-49ec-8d89-50a1eb9105e8 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:16:40.011109 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a565db12-9004-4580-8743-d3e672ac200c service cinder] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:16:40.038615 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1285/2555] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:16:39 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 15:16:40.039989 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1271/2556] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 15:16:39 2022] PUT /identity/v3/projects/fef4bb5baec149eeae8fa64e52ee0f97/users/b32c02ae04e5439aabf9d0f78c3868c9/roles/a2e551773cbd4df084020676865b8222 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 15:16:40.043985 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a565db12-9004-4580-8743-d3e672ac200c service cinder] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:16:40.061574 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a565db12-9004-4580-8743-d3e672ac200c service cinder] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:16:40.143620 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ea1401-8a9f-4ac4-8367-43acbae88ead admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:16:40.144389 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ea1401-8a9f-4ac4-8367-43acbae88ead admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:16:40.144783 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ea1401-8a9f-4ac4-8367-43acbae88ead admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:16:40.145252 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ea1401-8a9f-4ac4-8367-43acbae88ead admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:16:40.146090 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ea1401-8a9f-4ac4-8367-43acbae88ead admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:16:40.148080 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ea1401-8a9f-4ac4-8367-43acbae88ead admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:16:40.169729 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1286/2557] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 15:16:40 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 15:16:40.171044 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b71e290-8b16-49ec-8d89-50a1eb9105e8 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:16:40.192184 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b71e290-8b16-49ec-8d89-50a1eb9105e8 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:16:40.204785 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42775617-84bf-4c18-a799-9e384e0293e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:16:40.207631 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42775617-84bf-4c18-a799-9e384e0293e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:16:40.207927 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42775617-84bf-4c18-a799-9e384e0293e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:16:40.208209 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42775617-84bf-4c18-a799-9e384e0293e5 admin admin] PATH_INFO: `/v3/projects/fef4bb5baec149eeae8fa64e52ee0f97/users/b32c02ae04e5439aabf9d0f78c3868c9/roles/a2e551773cbd4df084020676865b8222` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:16:40.208336 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ea1401-8a9f-4ac4-8367-43acbae88ead admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:16:40.212695 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42775617-84bf-4c18-a799-9e384e0293e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fef4bb5baec149eeae8fa64e52ee0f97, user_id=b32c02ae04e5439aabf9d0f78c3868c9, role_id=a2e551773cbd4df084020676865b8222)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:16:40.216057 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42775617-84bf-4c18-a799-9e384e0293e5 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:16:40.223525 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ea1401-8a9f-4ac4-8367-43acbae88ead admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:16:40.231613 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e615b5-8501-401e-a6bb-f158157712a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:16:40.233570 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1272/2558] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 15:16:40 2022] PUT /identity/v3/projects/fef4bb5baec149eeae8fa64e52ee0f97/users/b32c02ae04e5439aabf9d0f78c3868c9/roles/a2e551773cbd4df084020676865b8222 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:16:40.234555 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e615b5-8501-401e-a6bb-f158157712a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:16:40.234999 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e615b5-8501-401e-a6bb-f158157712a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:16:40.235420 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e615b5-8501-401e-a6bb-f158157712a3 admin admin] PATH_INFO: `/v3/users/d252ecc41fa343098a855a55323c6bf7` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:16:40.237383 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e615b5-8501-401e-a6bb-f158157712a3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d252ecc41fa343098a855a55323c6bf7)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:16:40.242536 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e615b5-8501-401e-a6bb-f158157712a3 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:16:40.245823 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643ea5c8-232e-4070-8bdf-c24731b69a71 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:16:40.246235 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643ea5c8-232e-4070-8bdf-c24731b69a71 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:16:40.246605 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643ea5c8-232e-4070-8bdf-c24731b69a71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:16:40.281151 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-643ea5c8-232e-4070-8bdf-c24731b69a71 None None] MFA Rules not processed for user `b32c02ae04e5439aabf9d0f78c3868c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:16:40.284130 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-643ea5c8-232e-4070-8bdf-c24731b69a71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:16:40.346075 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-b5e615b5-8501-401e-a6bb-f158157712a3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd252ecc41fa343098a855a55323c6bf7'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:16:40.349303 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:16:40.361877 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-b5e615b5-8501-401e-a6bb-f158157712a3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd252ecc41fa343098a855a55323c6bf7'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:16:40.362906 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-b5e615b5-8501-401e-a6bb-f158157712a3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd252ecc41fa343098a855a55323c6bf7'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:16:40.385688 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-b5e615b5-8501-401e-a6bb-f158157712a3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd252ecc41fa343098a855a55323c6bf7'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:16:40.397694 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-b5e615b5-8501-401e-a6bb-f158157712a3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd252ecc41fa343098a855a55323c6bf7'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:16:40.406202 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1287/2559] 10.222.0.46 () {58 vars in 1223 bytes} [Thu May 26 15:16:40 2022] DELETE /identity/v3/users/d252ecc41fa343098a855a55323c6bf7 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:16:40.412350 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ca4e37-984e-49a0-bd3c-dcbc65c92bde None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:16:40.412350 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ca4e37-984e-49a0-bd3c-dcbc65c92bde None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:16:40.412350 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ca4e37-984e-49a0-bd3c-dcbc65c92bde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:16:40.413151 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1273/2560] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:16:40 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:16:40.430605 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-e4ca4e37-984e-49a0-bd3c-dcbc65c92bde None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:16:40.431871 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643ea5c8-232e-4070-8bdf-c24731b69a71 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:16:40.432921 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-e4ca4e37-984e-49a0-bd3c-dcbc65c92bde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:16:40.433598 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-643ea5c8-232e-4070-8bdf-c24731b69a71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:16:40.472252 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:16:40.552204 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1288/2561] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:16:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 15:16:40.563044 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ca4e37-984e-49a0-bd3c-dcbc65c92bde None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:16:40.565488 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-e4ca4e37-984e-49a0-bd3c-dcbc65c92bde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:16:40.579639 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643ea5c8-232e-4070-8bdf-c24731b69a71 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:16:40.715249 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa042d6c-89c0-4ef4-a88f-0e473701296b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5b5b35614a547d783e6e77ddb8c1c03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:16:40.717820 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa042d6c-89c0-4ef4-a88f-0e473701296b service cinder] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:16:40.718097 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ca4e37-984e-49a0-bd3c-dcbc65c92bde None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:16:40.718773 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa042d6c-89c0-4ef4-a88f-0e473701296b service cinder] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:16:40.719911 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa042d6c-89c0-4ef4-a88f-0e473701296b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:16:40.728769 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bede84-2f49-4310-a60d-7ab3ff4629f0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0156a2f6f3794f479e76b85bdbbe5c26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:16:40.729495 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bede84-2f49-4310-a60d-7ab3ff4629f0 service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:16:40.738664 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bede84-2f49-4310-a60d-7ab3ff4629f0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:16:40.738664 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bede84-2f49-4310-a60d-7ab3ff4629f0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:16:40.746550 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bede84-2f49-4310-a60d-7ab3ff4629f0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:16:40.748103 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bede84-2f49-4310-a60d-7ab3ff4629f0 service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:16:40.769592 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1289/2562] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:16:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 15:16:40.888774 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa042d6c-89c0-4ef4-a88f-0e473701296b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:16:40.893842 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa042d6c-89c0-4ef4-a88f-0e473701296b service cinder] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:16:40.925539 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1274/2563] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:16:40 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 495 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 15:16:41.204370 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bede84-2f49-4310-a60d-7ab3ff4629f0 service neutron] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:16:41.220506 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bede84-2f49-4310-a60d-7ab3ff4629f0 service neutron] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:16:41.228305 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6878b146-6708-463f-a399-95db923167eb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5b5b35614a547d783e6e77ddb8c1c03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:16:41.228972 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6878b146-6708-463f-a399-95db923167eb service cinder] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:16:41.229249 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6878b146-6708-463f-a399-95db923167eb service cinder] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:16:41.229565 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6878b146-6708-463f-a399-95db923167eb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:16:41.231442 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-6878b146-6708-463f-a399-95db923167eb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:16:41.275186 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa042d6c-89c0-4ef4-a88f-0e473701296b service cinder] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:16:41.278782 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-aa042d6c-89c0-4ef4-a88f-0e473701296b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:16:41.345602 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa042d6c-89c0-4ef4-a88f-0e473701296b service cinder] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:16:41.355608 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb1505e-2d6c-4fcd-a054-bb0da16d8ccc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:16:41.357163 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb1505e-2d6c-4fcd-a054-bb0da16d8ccc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:16:41.357742 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb1505e-2d6c-4fcd-a054-bb0da16d8ccc admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:16:41.358250 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb1505e-2d6c-4fcd-a054-bb0da16d8ccc admin admin] PATH_INFO: `/v3/projects/abd94b4320c746d7a97dd173addfdd8b` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:16:41.361100 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb1505e-2d6c-4fcd-a054-bb0da16d8ccc admin admin] RBAC: Authorizing `identity:delete_project(project_id=abd94b4320c746d7a97dd173addfdd8b)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:16:41.365210 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6878b146-6708-463f-a399-95db923167eb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:16:41.365793 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb1505e-2d6c-4fcd-a054-bb0da16d8ccc admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:16:41.367349 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6878b146-6708-463f-a399-95db923167eb service cinder] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:16:41.391002 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1290/2564] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:16:41 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 15:16:41.456745 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:16:41.490453 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-9cb1505e-2d6c-4fcd-a054-bb0da16d8ccc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'abd94b4320c746d7a97dd173addfdd8b', 'request_id': 'req-9cb1505e-2d6c-4fcd-a054-bb0da16d8ccc'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:16:41.506669 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-9cb1505e-2d6c-4fcd-a054-bb0da16d8ccc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'abd94b4320c746d7a97dd173addfdd8b', 'request_id': 'req-9cb1505e-2d6c-4fcd-a054-bb0da16d8ccc'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:16:41.526117 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-9cb1505e-2d6c-4fcd-a054-bb0da16d8ccc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'abd94b4320c746d7a97dd173addfdd8b', 'request_id': 'req-9cb1505e-2d6c-4fcd-a054-bb0da16d8ccc'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:16:41.566606 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-9cb1505e-2d6c-4fcd-a054-bb0da16d8ccc admin admin] The token cache is being invalidate because project abd94b4320c746d7a97dd173addfdd8b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 15:16:41.566903 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-9cb1505e-2d6c-4fcd-a054-bb0da16d8ccc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:16:41.569507 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-9cb1505e-2d6c-4fcd-a054-bb0da16d8ccc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:16:41.574607 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1275/2565] 10.222.0.46 () {58 vars in 1232 bytes} [Thu May 26 15:16:41 2022] DELETE /identity/v3/projects/abd94b4320c746d7a97dd173addfdd8b => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:16:41.576929 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6878b146-6708-463f-a399-95db923167eb service cinder] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:16:41.595834 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6878b146-6708-463f-a399-95db923167eb service cinder] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:16:41.766922 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d900d083-d37a-4677-9bcd-5bb208692825 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:16:41.771101 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d900d083-d37a-4677-9bcd-5bb208692825 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:16:41.771392 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d900d083-d37a-4677-9bcd-5bb208692825 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:16:41.771779 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d900d083-d37a-4677-9bcd-5bb208692825 admin admin] PATH_INFO: `/v3/users/ed4151ab4624462395a3337ccc5e8e3e` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:16:41.776944 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d900d083-d37a-4677-9bcd-5bb208692825 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ed4151ab4624462395a3337ccc5e8e3e)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:16:41.781061 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d900d083-d37a-4677-9bcd-5bb208692825 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:16:41.820699 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:16:41.888587 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-d900d083-d37a-4677-9bcd-5bb208692825 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ed4151ab4624462395a3337ccc5e8e3e'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:16:41.902372 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-d900d083-d37a-4677-9bcd-5bb208692825 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ed4151ab4624462395a3337ccc5e8e3e'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:16:41.903693 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-d900d083-d37a-4677-9bcd-5bb208692825 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ed4151ab4624462395a3337ccc5e8e3e'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:16:41.938721 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-d900d083-d37a-4677-9bcd-5bb208692825 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ed4151ab4624462395a3337ccc5e8e3e'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:16:41.979059 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-d900d083-d37a-4677-9bcd-5bb208692825 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ed4151ab4624462395a3337ccc5e8e3e'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:16:42.019993 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1291/2566] 10.222.0.46 () {58 vars in 1223 bytes} [Thu May 26 15:16:41 2022] DELETE /identity/v3/users/ed4151ab4624462395a3337ccc5e8e3e => generated 0 bytes in 437 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:16:42.764087 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb1505e-2d6c-4fcd-a054-bb0da16d8ccc admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:16:42.765738 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-9cb1505e-2d6c-4fcd-a054-bb0da16d8ccc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:16:42.787908 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb1505e-2d6c-4fcd-a054-bb0da16d8ccc admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:16:42.934618 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d41121-49ad-4991-9c7a-0f3dc8855935 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:16:42.935865 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d41121-49ad-4991-9c7a-0f3dc8855935 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:16:42.936351 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d41121-49ad-4991-9c7a-0f3dc8855935 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:16:42.936683 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d41121-49ad-4991-9c7a-0f3dc8855935 admin admin] PATH_INFO: `/v3/projects/5931da338ccd433c988ebe5f6ccdc076` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:16:42.939138 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d41121-49ad-4991-9c7a-0f3dc8855935 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5931da338ccd433c988ebe5f6ccdc076)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:16:42.941235 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d41121-49ad-4991-9c7a-0f3dc8855935 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:16:43.087963 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-95d41121-49ad-4991-9c7a-0f3dc8855935 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5931da338ccd433c988ebe5f6ccdc076', 'request_id': 'req-95d41121-49ad-4991-9c7a-0f3dc8855935'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:16:43.100117 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-95d41121-49ad-4991-9c7a-0f3dc8855935 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5931da338ccd433c988ebe5f6ccdc076', 'request_id': 'req-95d41121-49ad-4991-9c7a-0f3dc8855935'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:16:43.116245 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-95d41121-49ad-4991-9c7a-0f3dc8855935 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5931da338ccd433c988ebe5f6ccdc076', 'request_id': 'req-95d41121-49ad-4991-9c7a-0f3dc8855935'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:16:43.161635 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-95d41121-49ad-4991-9c7a-0f3dc8855935 admin admin] The token cache is being invalidate because project 5931da338ccd433c988ebe5f6ccdc076 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 15:16:43.162019 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-95d41121-49ad-4991-9c7a-0f3dc8855935 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:16:43.164822 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-95d41121-49ad-4991-9c7a-0f3dc8855935 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:16:43.168527 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1276/2567] 10.222.0.46 () {58 vars in 1232 bytes} [Thu May 26 15:16:42 2022] DELETE /identity/v3/projects/5931da338ccd433c988ebe5f6ccdc076 => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:16:43.174738 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c403ae-db1e-4418-bb16-dc6a621688db None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:16:43.180020 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c403ae-db1e-4418-bb16-dc6a621688db None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:16:43.180666 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c403ae-db1e-4418-bb16-dc6a621688db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:16:43.237091 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-49c403ae-db1e-4418-bb16-dc6a621688db None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:16:43.238592 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-49c403ae-db1e-4418-bb16-dc6a621688db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:16:43.402039 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1292/2568] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:16:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:16:43.412660 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef22a2d-b31a-4314-8974-b8aecefd7c52 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:16:43.413084 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef22a2d-b31a-4314-8974-b8aecefd7c52 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:16:43.413441 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef22a2d-b31a-4314-8974-b8aecefd7c52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:16:43.458874 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-4ef22a2d-b31a-4314-8974-b8aecefd7c52 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:16:43.460533 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-4ef22a2d-b31a-4314-8974-b8aecefd7c52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:16:43.505952 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1277/2569] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:16:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:16:43.510650 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c403ae-db1e-4418-bb16-dc6a621688db None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:16:43.527563 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c403ae-db1e-4418-bb16-dc6a621688db None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:16:43.549368 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ea9e22-d772-4193-b1fa-1da12fb9e679 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:16:43.550898 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ea9e22-d772-4193-b1fa-1da12fb9e679 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:16:43.552485 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ea9e22-d772-4193-b1fa-1da12fb9e679 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:16:43.552485 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ea9e22-d772-4193-b1fa-1da12fb9e679 admin admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:16:43.555674 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ea9e22-d772-4193-b1fa-1da12fb9e679 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 15:16:43.559469 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ea9e22-d772-4193-b1fa-1da12fb9e679 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:16:43.566000 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ea9e22-d772-4193-b1fa-1da12fb9e679 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:16:43.626818 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1293/2570] 10.222.0.46 () {58 vars in 1152 bytes} [Thu May 26 15:16:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 15:16:43.628182 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef22a2d-b31a-4314-8974-b8aecefd7c52 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:16:43.674643 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef22a2d-b31a-4314-8974-b8aecefd7c52 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:16:43.695321 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b9a2aa-cbc2-49c6-b764-5b958a4457d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:16:43.699388 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b9a2aa-cbc2-49c6-b764-5b958a4457d5 admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:16:43.701247 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b9a2aa-cbc2-49c6-b764-5b958a4457d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:16:43.701656 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b9a2aa-cbc2-49c6-b764-5b958a4457d5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:16:43.702473 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b9a2aa-cbc2-49c6-b764-5b958a4457d5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:16:43.709923 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b9a2aa-cbc2-49c6-b764-5b958a4457d5 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:16:43.775886 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1278/2571] 10.222.0.46 () {60 vars in 1152 bytes} [Thu May 26 15:16:43 2022] POST /identity/v3/projects => generated 383 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 15:16:43.781496 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ea9e22-d772-4193-b1fa-1da12fb9e679 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:16:43.793089 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ea9e22-d772-4193-b1fa-1da12fb9e679 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:16:43.934860 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185b058a-23be-4884-8d11-f404f6186d34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:16:43.935647 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185b058a-23be-4884-8d11-f404f6186d34 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:16:43.935999 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185b058a-23be-4884-8d11-f404f6186d34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:16:43.936342 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185b058a-23be-4884-8d11-f404f6186d34 admin admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:16:43.937276 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185b058a-23be-4884-8d11-f404f6186d34 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:16:43.939401 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185b058a-23be-4884-8d11-f404f6186d34 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:16:43.975351 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1294/2572] 10.222.0.46 () {60 vars in 1143 bytes} [Thu May 26 15:16:43 2022] POST /identity/v3/users => generated 292 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 15:16:43.979169 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b9a2aa-cbc2-49c6-b764-5b958a4457d5 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:16:43.993272 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b9a2aa-cbc2-49c6-b764-5b958a4457d5 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:16:44.001745 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f7c24e-3b39-4b52-82ec-c783c3f515a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:16:44.002531 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f7c24e-3b39-4b52-82ec-c783c3f515a7 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:16:44.002912 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f7c24e-3b39-4b52-82ec-c783c3f515a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:16:44.003240 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f7c24e-3b39-4b52-82ec-c783c3f515a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:16:44.004048 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f7c24e-3b39-4b52-82ec-c783c3f515a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:16:44.006803 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f7c24e-3b39-4b52-82ec-c783c3f515a7 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:16:44.021358 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1279/2573] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 15:16:43 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 15:16:44.025588 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185b058a-23be-4884-8d11-f404f6186d34 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:16:44.037531 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185b058a-23be-4884-8d11-f404f6186d34 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:16:44.043925 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b298f63-2ad0-416c-9e91-d138d3ae3ddc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:16:44.044738 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b298f63-2ad0-416c-9e91-d138d3ae3ddc admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:16:44.045140 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b298f63-2ad0-416c-9e91-d138d3ae3ddc admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:16:44.045516 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b298f63-2ad0-416c-9e91-d138d3ae3ddc admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:16:44.046338 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b298f63-2ad0-416c-9e91-d138d3ae3ddc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:16:44.049679 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b298f63-2ad0-416c-9e91-d138d3ae3ddc admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:16:44.057473 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:DELETE=9,keystone:SELECT=87,keystone:INSERT=4 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:16:44.061912 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1295/2574] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 15:16:44 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 15:16:44.066128 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f7c24e-3b39-4b52-82ec-c783c3f515a7 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:16:44.080684 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f7c24e-3b39-4b52-82ec-c783c3f515a7 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:16:44.085860 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07828a0a-137a-4a1f-95c2-5c83b8666ad4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:16:44.086869 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07828a0a-137a-4a1f-95c2-5c83b8666ad4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:16:44.087151 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07828a0a-137a-4a1f-95c2-5c83b8666ad4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:16:44.087436 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07828a0a-137a-4a1f-95c2-5c83b8666ad4 admin admin] PATH_INFO: `/v3/projects/ce2eb525209a489aab7474578f33788f/users/c3979e46a9ac440ba8ff39c8e1d7a7e0/roles/d4088d1845a24fbd9bd8975ce0b1d3d0` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:16:44.092317 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:16:44.111354 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07828a0a-137a-4a1f-95c2-5c83b8666ad4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce2eb525209a489aab7474578f33788f, user_id=c3979e46a9ac440ba8ff39c8e1d7a7e0, role_id=d4088d1845a24fbd9bd8975ce0b1d3d0)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:16:44.114288 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07828a0a-137a-4a1f-95c2-5c83b8666ad4 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:16:44.139936 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1280/2575] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 15:16:44 2022] PUT /identity/v3/projects/ce2eb525209a489aab7474578f33788f/users/c3979e46a9ac440ba8ff39c8e1d7a7e0/roles/d4088d1845a24fbd9bd8975ce0b1d3d0 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 15:16:44.150721 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ab9a87-330f-4048-8f1f-a14a8d2d1d5b None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:16:44.151059 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ab9a87-330f-4048-8f1f-a14a8d2d1d5b None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:16:44.151342 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ab9a87-330f-4048-8f1f-a14a8d2d1d5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:16:44.185698 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-a7ab9a87-330f-4048-8f1f-a14a8d2d1d5b None None] MFA Rules not processed for user `c3979e46a9ac440ba8ff39c8e1d7a7e0`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:16:44.186948 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-a7ab9a87-330f-4048-8f1f-a14a8d2d1d5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:16:44.318041 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1296/2576] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:16:44 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:16:44.329705 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07828a0a-137a-4a1f-95c2-5c83b8666ad4 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:16:44.331783 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-07828a0a-137a-4a1f-95c2-5c83b8666ad4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:16:44.413926 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07828a0a-137a-4a1f-95c2-5c83b8666ad4 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:16:44.418366 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9270e2-7bcc-475c-b4e1-41d205614e7f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5b5b35614a547d783e6e77ddb8c1c03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:16:44.419059 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9270e2-7bcc-475c-b4e1-41d205614e7f service cinder] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:16:44.419341 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9270e2-7bcc-475c-b4e1-41d205614e7f service cinder] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:16:44.419633 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9270e2-7bcc-475c-b4e1-41d205614e7f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:16:44.429296 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9270e2-7bcc-475c-b4e1-41d205614e7f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:16:44.430851 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf9270e2-7bcc-475c-b4e1-41d205614e7f service cinder] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:16:44.442254 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1281/2577] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:16:44 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 15:16:44.469587 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ab9a87-330f-4048-8f1f-a14a8d2d1d5b None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:16:44.559192 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ab9a87-330f-4048-8f1f-a14a8d2d1d5b None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:16:44.564576 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d25443-2f67-46fb-bdc6-121c816e3891 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:16:44.565595 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d25443-2f67-46fb-bdc6-121c816e3891 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:16:44.565882 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d25443-2f67-46fb-bdc6-121c816e3891 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:16:44.566162 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d25443-2f67-46fb-bdc6-121c816e3891 admin admin] PATH_INFO: `/v3/users/c3979e46a9ac440ba8ff39c8e1d7a7e0` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:16:44.567842 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d25443-2f67-46fb-bdc6-121c816e3891 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c3979e46a9ac440ba8ff39c8e1d7a7e0)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:16:44.570405 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d25443-2f67-46fb-bdc6-121c816e3891 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:16:44.716260 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-f8d25443-2f67-46fb-bdc6-121c816e3891 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c3979e46a9ac440ba8ff39c8e1d7a7e0'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:16:44.728388 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-f8d25443-2f67-46fb-bdc6-121c816e3891 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c3979e46a9ac440ba8ff39c8e1d7a7e0'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:16:44.729242 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-f8d25443-2f67-46fb-bdc6-121c816e3891 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c3979e46a9ac440ba8ff39c8e1d7a7e0'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:16:44.749731 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-f8d25443-2f67-46fb-bdc6-121c816e3891 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c3979e46a9ac440ba8ff39c8e1d7a7e0'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:16:44.758647 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-f8d25443-2f67-46fb-bdc6-121c816e3891 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c3979e46a9ac440ba8ff39c8e1d7a7e0'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:16:44.770334 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1297/2578] 10.222.0.46 () {58 vars in 1223 bytes} [Thu May 26 15:16:44 2022] DELETE /identity/v3/users/c3979e46a9ac440ba8ff39c8e1d7a7e0 => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:16:44.774697 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a8794c-62c3-4560-bab1-4a616b58f09d None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:16:44.775258 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a8794c-62c3-4560-bab1-4a616b58f09d None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:16:44.775542 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a8794c-62c3-4560-bab1-4a616b58f09d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:16:44.804815 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-a5a8794c-62c3-4560-bab1-4a616b58f09d None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:16:44.808537 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-a5a8794c-62c3-4560-bab1-4a616b58f09d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:16:44.916752 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1282/2579] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:16:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:16:44.927684 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d25443-2f67-46fb-bdc6-121c816e3891 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:16:44.929166 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-f8d25443-2f67-46fb-bdc6-121c816e3891 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:16:44.942525 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:16:45.023141 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d25443-2f67-46fb-bdc6-121c816e3891 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:16:45.031089 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d77999-d803-4370-8829-c7af1ef3654f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0156a2f6f3794f479e76b85bdbbe5c26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:16:45.031782 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d77999-d803-4370-8829-c7af1ef3654f service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:16:45.032068 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d77999-d803-4370-8829-c7af1ef3654f service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:16:45.032349 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d77999-d803-4370-8829-c7af1ef3654f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:16:45.047737 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d77999-d803-4370-8829-c7af1ef3654f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:16:45.049211 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d77999-d803-4370-8829-c7af1ef3654f service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:16:45.060202 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1298/2580] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:16:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 15:16:45.479640 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a8794c-62c3-4560-bab1-4a616b58f09d None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:16:45.481133 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-a5a8794c-62c3-4560-bab1-4a616b58f09d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:16:45.505250 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a8794c-62c3-4560-bab1-4a616b58f09d None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:16:45.518046 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59d35c1-4a11-4311-a517-8a9b489c86b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:16:45.518614 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59d35c1-4a11-4311-a517-8a9b489c86b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:16:45.518958 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59d35c1-4a11-4311-a517-8a9b489c86b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:16:45.519253 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59d35c1-4a11-4311-a517-8a9b489c86b1 admin admin] PATH_INFO: `/v3/projects/ce2eb525209a489aab7474578f33788f` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:16:45.520694 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59d35c1-4a11-4311-a517-8a9b489c86b1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ce2eb525209a489aab7474578f33788f)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:16:45.522504 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59d35c1-4a11-4311-a517-8a9b489c86b1 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:16:45.616571 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-c59d35c1-4a11-4311-a517-8a9b489c86b1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ce2eb525209a489aab7474578f33788f', 'request_id': 'req-c59d35c1-4a11-4311-a517-8a9b489c86b1'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:16:45.624752 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-c59d35c1-4a11-4311-a517-8a9b489c86b1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ce2eb525209a489aab7474578f33788f', 'request_id': 'req-c59d35c1-4a11-4311-a517-8a9b489c86b1'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:16:45.637252 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-c59d35c1-4a11-4311-a517-8a9b489c86b1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ce2eb525209a489aab7474578f33788f', 'request_id': 'req-c59d35c1-4a11-4311-a517-8a9b489c86b1'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:16:45.658697 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-c59d35c1-4a11-4311-a517-8a9b489c86b1 admin admin] The token cache is being invalidate because project ce2eb525209a489aab7474578f33788f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 15:16:45.659260 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-c59d35c1-4a11-4311-a517-8a9b489c86b1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:16:45.659979 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-c59d35c1-4a11-4311-a517-8a9b489c86b1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:16:45.663135 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1283/2581] 10.222.0.46 () {58 vars in 1232 bytes} [Thu May 26 15:16:45 2022] DELETE /identity/v3/projects/ce2eb525209a489aab7474578f33788f => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:16:45.670883 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6511c6ab-1d24-4e47-b8be-0d818960a002 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:16:45.671365 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6511c6ab-1d24-4e47-b8be-0d818960a002 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:16:45.674234 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6511c6ab-1d24-4e47-b8be-0d818960a002 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:16:45.738879 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-6511c6ab-1d24-4e47-b8be-0d818960a002 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:16:45.739502 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-6511c6ab-1d24-4e47-b8be-0d818960a002 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:16:45.832946 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1299/2582] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:16:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:16:45.841134 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362ad127-5994-4b75-be25-d8d9592bb5dc None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:16:45.841652 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362ad127-5994-4b75-be25-d8d9592bb5dc None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:16:45.841736 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362ad127-5994-4b75-be25-d8d9592bb5dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:16:45.866610 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-362ad127-5994-4b75-be25-d8d9592bb5dc None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:16:45.868495 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-362ad127-5994-4b75-be25-d8d9592bb5dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:16:45.878916 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1284/2583] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:16:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:16:45.883337 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6511c6ab-1d24-4e47-b8be-0d818960a002 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:16:45.896678 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6511c6ab-1d24-4e47-b8be-0d818960a002 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:16:45.901893 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50700d5a-589c-4743-b773-b2e31bc4afb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:16:45.902904 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50700d5a-589c-4743-b773-b2e31bc4afb9 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:16:45.903190 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50700d5a-589c-4743-b773-b2e31bc4afb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:16:45.903527 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50700d5a-589c-4743-b773-b2e31bc4afb9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:16:45.904277 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50700d5a-589c-4743-b773-b2e31bc4afb9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 15:16:45.904617 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50700d5a-589c-4743-b773-b2e31bc4afb9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:16:45.908132 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50700d5a-589c-4743-b773-b2e31bc4afb9 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:16:45.920571 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1300/2584] 10.222.0.46 () {58 vars in 1152 bytes} [Thu May 26 15:16:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 15:16:45.925054 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362ad127-5994-4b75-be25-d8d9592bb5dc None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:16:45.931303 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:16:45.935553 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362ad127-5994-4b75-be25-d8d9592bb5dc None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:16:45.942344 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66d9eec-1baa-476a-9c43-1bd811cb0cf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:16:45.943429 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66d9eec-1baa-476a-9c43-1bd811cb0cf7 admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:16:45.944344 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66d9eec-1baa-476a-9c43-1bd811cb0cf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:16:45.944653 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66d9eec-1baa-476a-9c43-1bd811cb0cf7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:16:45.945690 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66d9eec-1baa-476a-9c43-1bd811cb0cf7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:16:45.948838 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66d9eec-1baa-476a-9c43-1bd811cb0cf7 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:16:45.989229 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1285/2585] 10.222.0.46 () {60 vars in 1152 bytes} [Thu May 26 15:16:45 2022] POST /identity/v3/projects => generated 371 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 15:16:45.990516 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50700d5a-589c-4743-b773-b2e31bc4afb9 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:16:46.003116 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50700d5a-589c-4743-b773-b2e31bc4afb9 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:16:46.083943 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cfe388-cc0c-4bda-b304-ac54bae25e68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:16:46.084668 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cfe388-cc0c-4bda-b304-ac54bae25e68 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:16:46.084960 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cfe388-cc0c-4bda-b304-ac54bae25e68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:16:46.085290 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cfe388-cc0c-4bda-b304-ac54bae25e68 admin admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:16:46.086044 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54cfe388-cc0c-4bda-b304-ac54bae25e68 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:16:46.088129 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54cfe388-cc0c-4bda-b304-ac54bae25e68 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:16:46.114245 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1301/2586] 10.222.0.46 () {60 vars in 1143 bytes} [Thu May 26 15:16:45 2022] POST /identity/v3/users => generated 286 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 15:16:46.119152 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66d9eec-1baa-476a-9c43-1bd811cb0cf7 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:16:46.130427 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66d9eec-1baa-476a-9c43-1bd811cb0cf7 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:16:46.138319 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b0ad8d-de27-4f23-ae4a-1a927e798e3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:16:46.139309 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b0ad8d-de27-4f23-ae4a-1a927e798e3c admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:16:46.140071 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b0ad8d-de27-4f23-ae4a-1a927e798e3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:16:46.140633 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b0ad8d-de27-4f23-ae4a-1a927e798e3c admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:16:46.141525 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b0ad8d-de27-4f23-ae4a-1a927e798e3c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:16:46.143552 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b0ad8d-de27-4f23-ae4a-1a927e798e3c admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:16:46.157826 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1286/2587] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 15:16:46 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 15:16:46.162121 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cfe388-cc0c-4bda-b304-ac54bae25e68 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:16:46.172939 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cfe388-cc0c-4bda-b304-ac54bae25e68 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:16:46.182995 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e950a671-fb68-4308-b3c0-9a06bf523932 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:16:46.183759 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e950a671-fb68-4308-b3c0-9a06bf523932 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:16:46.184022 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e950a671-fb68-4308-b3c0-9a06bf523932 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:16:46.184312 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e950a671-fb68-4308-b3c0-9a06bf523932 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:16:46.185091 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e950a671-fb68-4308-b3c0-9a06bf523932 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:16:46.186939 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e950a671-fb68-4308-b3c0-9a06bf523932 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:16:46.205556 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1302/2588] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 15:16:46 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 15:16:46.206757 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b0ad8d-de27-4f23-ae4a-1a927e798e3c admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:16:46.218631 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b0ad8d-de27-4f23-ae4a-1a927e798e3c admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:16:46.223517 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1b7062-4684-44f8-9d68-46021c73ae41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:16:46.224413 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1b7062-4684-44f8-9d68-46021c73ae41 admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:16:46.224697 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1b7062-4684-44f8-9d68-46021c73ae41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:16:46.224978 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1b7062-4684-44f8-9d68-46021c73ae41 admin admin] PATH_INFO: `/v3/projects/d2169b075ad8405fbbf81d8fc39b0b53/users/cd74387e84d042c0a0bc5b151df3f6f6/roles/d4088d1845a24fbd9bd8975ce0b1d3d0` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:16:46.242713 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1b7062-4684-44f8-9d68-46021c73ae41 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d2169b075ad8405fbbf81d8fc39b0b53, user_id=cd74387e84d042c0a0bc5b151df3f6f6, role_id=d4088d1845a24fbd9bd8975ce0b1d3d0)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:16:46.247009 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1b7062-4684-44f8-9d68-46021c73ae41 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:16:46.266276 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1287/2589] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 15:16:46 2022] PUT /identity/v3/projects/d2169b075ad8405fbbf81d8fc39b0b53/users/cd74387e84d042c0a0bc5b151df3f6f6/roles/d4088d1845a24fbd9bd8975ce0b1d3d0 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:16:46.282326 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee8b94d-f87f-4e32-9461-a702a52e0427 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:16:46.282946 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee8b94d-f87f-4e32-9461-a702a52e0427 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:16:46.283343 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee8b94d-f87f-4e32-9461-a702a52e0427 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:16:46.326964 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-bee8b94d-f87f-4e32-9461-a702a52e0427 None None] MFA Rules not processed for user `cd74387e84d042c0a0bc5b151df3f6f6`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:16:46.328822 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-bee8b94d-f87f-4e32-9461-a702a52e0427 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:16:46.336969 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:16:46.441915 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1303/2590] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:16:46 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:16:46.459823 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1b7062-4684-44f8-9d68-46021c73ae41 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:16:46.462779 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-3a1b7062-4684-44f8-9d68-46021c73ae41 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:16:46.545554 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1b7062-4684-44f8-9d68-46021c73ae41 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:16:46.549593 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ad793f-d658-48d5-99a4-0eb9c244023d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a436dbdee624f44ad76cca8416b0ad1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:16:46.550322 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ad793f-d658-48d5-99a4-0eb9c244023d service glance] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:16:46.550648 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ad793f-d658-48d5-99a4-0eb9c244023d service glance] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:16:46.550976 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ad793f-d658-48d5-99a4-0eb9c244023d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:16:46.561280 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ad793f-d658-48d5-99a4-0eb9c244023d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:16:46.562776 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ad793f-d658-48d5-99a4-0eb9c244023d service glance] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:16:46.587009 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1288/2591] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:16:46 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 15:16:49.856007 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73ab915-00f4-4b42-bcda-105b93820660 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:16:49.856880 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73ab915-00f4-4b42-bcda-105b93820660 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:16:49.856880 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73ab915-00f4-4b42-bcda-105b93820660 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:16:49.880306 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-e73ab915-00f4-4b42-bcda-105b93820660 None None] MFA Rules not processed for user `e6e2622df88f4e1b9b81097d6e558f21`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:16:49.881692 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-e73ab915-00f4-4b42-bcda-105b93820660 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:16:49.948786 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1304/2592] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:16:49 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:16:50.231590 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ad793f-d658-48d5-99a4-0eb9c244023d service glance] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:16:50.233106 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-f7ad793f-d658-48d5-99a4-0eb9c244023d service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:16:50.295212 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ad793f-d658-48d5-99a4-0eb9c244023d service glance] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:16:50.299134 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6012a21f-9b1b-4e0d-b1a9-a56fc6d6d0fb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0156a2f6f3794f479e76b85bdbbe5c26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:16:50.299813 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6012a21f-9b1b-4e0d-b1a9-a56fc6d6d0fb service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:16:50.300100 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6012a21f-9b1b-4e0d-b1a9-a56fc6d6d0fb service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:16:50.300513 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6012a21f-9b1b-4e0d-b1a9-a56fc6d6d0fb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:16:50.302355 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-6012a21f-9b1b-4e0d-b1a9-a56fc6d6d0fb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:16:50.313085 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6012a21f-9b1b-4e0d-b1a9-a56fc6d6d0fb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:16:50.314590 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6012a21f-9b1b-4e0d-b1a9-a56fc6d6d0fb service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:16:50.323956 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1289/2593] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:16:50 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 15:16:52.142387 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db75ada-aa99-4ed2-8828-cd34c797d1be None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:16:52.143263 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db75ada-aa99-4ed2-8828-cd34c797d1be None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:16:52.143577 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db75ada-aa99-4ed2-8828-cd34c797d1be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:16:52.175801 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-9db75ada-aa99-4ed2-8828-cd34c797d1be None None] MFA Rules not processed for user `8028c520d93449a6a087c281280ee13d`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:16:52.177559 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-9db75ada-aa99-4ed2-8828-cd34c797d1be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:16:52.258396 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1305/2594] 10.222.0.46 () {58 vars in 1009 bytes} [Thu May 26 15:16:52 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:16:52.270485 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6012a21f-9b1b-4e0d-b1a9-a56fc6d6d0fb service neutron] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:16:52.272272 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-6012a21f-9b1b-4e0d-b1a9-a56fc6d6d0fb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:16:52.343997 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:16:52.378141 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6012a21f-9b1b-4e0d-b1a9-a56fc6d6d0fb service neutron] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:16:52.383315 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b064d096-a908-4da6-b95f-bf65918b9516 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4bea272624b45d8ac2ad27bd07df885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:16:52.383988 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b064d096-a908-4da6-b95f-bf65918b9516 service swift] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:16:52.384272 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b064d096-a908-4da6-b95f-bf65918b9516 service swift] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:16:52.384548 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b064d096-a908-4da6-b95f-bf65918b9516 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:16:52.396234 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b064d096-a908-4da6-b95f-bf65918b9516 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:16:52.397921 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b064d096-a908-4da6-b95f-bf65918b9516 service swift] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:16:52.408163 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1290/2595] 10.222.0.46 () {60 vars in 1384 bytes} [Thu May 26 15:16:52 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 26 15:16:53.192300 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db75ada-aa99-4ed2-8828-cd34c797d1be None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:16:53.195274 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-9db75ada-aa99-4ed2-8828-cd34c797d1be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:16:53.334250 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:16:53.341338 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db75ada-aa99-4ed2-8828-cd34c797d1be None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:16:53.347863 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fffcd0-e97f-491d-bcbe-76c57a4fac99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:16:53.348779 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fffcd0-e97f-491d-bcbe-76c57a4fac99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:16:53.349166 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fffcd0-e97f-491d-bcbe-76c57a4fac99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:16:53.349674 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fffcd0-e97f-491d-bcbe-76c57a4fac99 admin admin] PATH_INFO: `/v3/users/aab211eb3c664ece82f090ce0cfc6658` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:16:53.351250 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fffcd0-e97f-491d-bcbe-76c57a4fac99 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aab211eb3c664ece82f090ce0cfc6658)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:16:53.354132 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49fffcd0-e97f-491d-bcbe-76c57a4fac99 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:16:53.479919 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-49fffcd0-e97f-491d-bcbe-76c57a4fac99 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aab211eb3c664ece82f090ce0cfc6658'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:16:53.506488 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-49fffcd0-e97f-491d-bcbe-76c57a4fac99 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aab211eb3c664ece82f090ce0cfc6658'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:16:53.507443 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-49fffcd0-e97f-491d-bcbe-76c57a4fac99 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aab211eb3c664ece82f090ce0cfc6658'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:16:53.559361 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-49fffcd0-e97f-491d-bcbe-76c57a4fac99 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aab211eb3c664ece82f090ce0cfc6658'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:16:53.586802 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-49fffcd0-e97f-491d-bcbe-76c57a4fac99 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aab211eb3c664ece82f090ce0cfc6658'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:16:53.603957 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1306/2596] 10.222.0.46 () {58 vars in 1223 bytes} [Thu May 26 15:16:53 2022] DELETE /identity/v3/users/aab211eb3c664ece82f090ce0cfc6658 => generated 0 bytes in 409 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:16:53.613038 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b99b6c-f46f-4d52-b0d2-37be2908279b None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:16:53.613637 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b99b6c-f46f-4d52-b0d2-37be2908279b None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:16:53.613637 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b99b6c-f46f-4d52-b0d2-37be2908279b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:16:53.663638 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-10b99b6c-f46f-4d52-b0d2-37be2908279b None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:16:53.665708 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-10b99b6c-f46f-4d52-b0d2-37be2908279b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:16:53.892582 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1291/2597] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:16:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 280 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:16:53.905085 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fffcd0-e97f-491d-bcbe-76c57a4fac99 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:16:53.908143 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-49fffcd0-e97f-491d-bcbe-76c57a4fac99 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:16:54.073013 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fffcd0-e97f-491d-bcbe-76c57a4fac99 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:16:54.089620 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda00b9a-6728-4746-b916-d62ef08b0f58 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0156a2f6f3794f479e76b85bdbbe5c26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:16:54.090462 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda00b9a-6728-4746-b916-d62ef08b0f58 service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:16:54.090806 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda00b9a-6728-4746-b916-d62ef08b0f58 service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:16:54.091184 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda00b9a-6728-4746-b916-d62ef08b0f58 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:16:54.144807 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda00b9a-6728-4746-b916-d62ef08b0f58 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:16:54.146271 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda00b9a-6728-4746-b916-d62ef08b0f58 service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:16:54.170803 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1307/2598] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:16:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 267 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 15:16:54.818819 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b99b6c-f46f-4d52-b0d2-37be2908279b None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:16:54.820683 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-10b99b6c-f46f-4d52-b0d2-37be2908279b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:16:54.856815 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b99b6c-f46f-4d52-b0d2-37be2908279b None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:16:54.889919 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65691a16-b862-4b5b-8aa6-e4b4b018b51a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:16:54.891402 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65691a16-b862-4b5b-8aa6-e4b4b018b51a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:16:54.892496 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65691a16-b862-4b5b-8aa6-e4b4b018b51a admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:16:54.892980 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65691a16-b862-4b5b-8aa6-e4b4b018b51a admin admin] PATH_INFO: `/v3/projects/0d40a0d3f8704d6bbc0f761581270ad7` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:16:54.896412 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65691a16-b862-4b5b-8aa6-e4b4b018b51a admin admin] RBAC: Authorizing `identity:delete_project(project_id=0d40a0d3f8704d6bbc0f761581270ad7)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:16:54.902412 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65691a16-b862-4b5b-8aa6-e4b4b018b51a admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:16:55.004726 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-65691a16-b862-4b5b-8aa6-e4b4b018b51a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0d40a0d3f8704d6bbc0f761581270ad7', 'request_id': 'req-65691a16-b862-4b5b-8aa6-e4b4b018b51a'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:16:55.023327 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-65691a16-b862-4b5b-8aa6-e4b4b018b51a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0d40a0d3f8704d6bbc0f761581270ad7', 'request_id': 'req-65691a16-b862-4b5b-8aa6-e4b4b018b51a'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:16:55.033770 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-65691a16-b862-4b5b-8aa6-e4b4b018b51a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0d40a0d3f8704d6bbc0f761581270ad7', 'request_id': 'req-65691a16-b862-4b5b-8aa6-e4b4b018b51a'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:16:55.093112 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-65691a16-b862-4b5b-8aa6-e4b4b018b51a admin admin] The token cache is being invalidate because project 0d40a0d3f8704d6bbc0f761581270ad7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 15:16:55.093684 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-65691a16-b862-4b5b-8aa6-e4b4b018b51a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:16:55.094624 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-65691a16-b862-4b5b-8aa6-e4b4b018b51a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:16:55.096802 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1292/2599] 10.222.0.46 () {58 vars in 1232 bytes} [Thu May 26 15:16:54 2022] DELETE /identity/v3/projects/0d40a0d3f8704d6bbc0f761581270ad7 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:16:55.100592 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda00b9a-6728-4746-b916-d62ef08b0f58 service neutron] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:16:55.115873 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda00b9a-6728-4746-b916-d62ef08b0f58 service neutron] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:16:55.326482 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62193c0b-3848-4bd7-8b4f-4d4913512d93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:16:55.326482 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62193c0b-3848-4bd7-8b4f-4d4913512d93 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:16:55.328110 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62193c0b-3848-4bd7-8b4f-4d4913512d93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:16:55.328110 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62193c0b-3848-4bd7-8b4f-4d4913512d93 admin admin] PATH_INFO: `/v3/users/b32c02ae04e5439aabf9d0f78c3868c9` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:16:55.334734 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62193c0b-3848-4bd7-8b4f-4d4913512d93 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b32c02ae04e5439aabf9d0f78c3868c9)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:16:55.350207 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62193c0b-3848-4bd7-8b4f-4d4913512d93 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:16:55.413097 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:16:55.495399 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-62193c0b-3848-4bd7-8b4f-4d4913512d93 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b32c02ae04e5439aabf9d0f78c3868c9'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:16:55.537890 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-62193c0b-3848-4bd7-8b4f-4d4913512d93 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b32c02ae04e5439aabf9d0f78c3868c9'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:16:55.539037 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-62193c0b-3848-4bd7-8b4f-4d4913512d93 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b32c02ae04e5439aabf9d0f78c3868c9'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:16:55.583919 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-62193c0b-3848-4bd7-8b4f-4d4913512d93 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b32c02ae04e5439aabf9d0f78c3868c9'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:16:55.596058 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-62193c0b-3848-4bd7-8b4f-4d4913512d93 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b32c02ae04e5439aabf9d0f78c3868c9'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:16:55.606867 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1308/2600] 10.222.0.46 () {58 vars in 1223 bytes} [Thu May 26 15:16:55 2022] DELETE /identity/v3/users/b32c02ae04e5439aabf9d0f78c3868c9 => generated 0 bytes in 505 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:16:56.310022 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65691a16-b862-4b5b-8aa6-e4b4b018b51a admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:16:56.313778 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-65691a16-b862-4b5b-8aa6-e4b4b018b51a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:16:56.360723 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65691a16-b862-4b5b-8aa6-e4b4b018b51a admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:16:56.554611 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4261f7-39e1-438d-89fe-dc7c5c13501e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:16:56.556382 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4261f7-39e1-438d-89fe-dc7c5c13501e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:16:56.557285 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4261f7-39e1-438d-89fe-dc7c5c13501e admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:16:56.557888 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4261f7-39e1-438d-89fe-dc7c5c13501e admin admin] PATH_INFO: `/v3/projects/fef4bb5baec149eeae8fa64e52ee0f97` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:16:56.560801 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4261f7-39e1-438d-89fe-dc7c5c13501e admin admin] RBAC: Authorizing `identity:delete_project(project_id=fef4bb5baec149eeae8fa64e52ee0f97)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:16:56.564822 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4261f7-39e1-438d-89fe-dc7c5c13501e admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:16:56.583988 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:16:56.691697 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-ba4261f7-39e1-438d-89fe-dc7c5c13501e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fef4bb5baec149eeae8fa64e52ee0f97', 'request_id': 'req-ba4261f7-39e1-438d-89fe-dc7c5c13501e'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:16:56.702479 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-ba4261f7-39e1-438d-89fe-dc7c5c13501e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fef4bb5baec149eeae8fa64e52ee0f97', 'request_id': 'req-ba4261f7-39e1-438d-89fe-dc7c5c13501e'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:16:56.713532 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-ba4261f7-39e1-438d-89fe-dc7c5c13501e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fef4bb5baec149eeae8fa64e52ee0f97', 'request_id': 'req-ba4261f7-39e1-438d-89fe-dc7c5c13501e'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:16:56.743323 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-ba4261f7-39e1-438d-89fe-dc7c5c13501e admin admin] The token cache is being invalidate because project fef4bb5baec149eeae8fa64e52ee0f97 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 15:16:56.743886 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-ba4261f7-39e1-438d-89fe-dc7c5c13501e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:16:56.745110 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-ba4261f7-39e1-438d-89fe-dc7c5c13501e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:16:56.749492 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1293/2601] 10.222.0.46 () {58 vars in 1232 bytes} [Thu May 26 15:16:56 2022] DELETE /identity/v3/projects/fef4bb5baec149eeae8fa64e52ee0f97 => generated 0 bytes in 440 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:16:56.756414 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a3650d-20a3-4c9c-8afc-5b6f95b21a88 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:16:56.757246 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a3650d-20a3-4c9c-8afc-5b6f95b21a88 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:16:56.757394 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a3650d-20a3-4c9c-8afc-5b6f95b21a88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:16:56.795507 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-c1a3650d-20a3-4c9c-8afc-5b6f95b21a88 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:16:56.799345 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-c1a3650d-20a3-4c9c-8afc-5b6f95b21a88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:16:56.936496 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1309/2602] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:16:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:16:56.947797 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f683267-f1ac-49f2-aba9-880db3025c6a None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:16:56.948150 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f683267-f1ac-49f2-aba9-880db3025c6a None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:16:56.948494 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f683267-f1ac-49f2-aba9-880db3025c6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:16:56.977743 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-5f683267-f1ac-49f2-aba9-880db3025c6a None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:16:56.979105 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-5f683267-f1ac-49f2-aba9-880db3025c6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:16:56.992320 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1294/2603] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:16:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 15:16:56.994375 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a3650d-20a3-4c9c-8afc-5b6f95b21a88 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:16:57.016331 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a3650d-20a3-4c9c-8afc-5b6f95b21a88 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:16:57.032537 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9a9e69-233c-46ff-9144-eef1f70445a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:16:57.033590 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9a9e69-233c-46ff-9144-eef1f70445a9 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:16:57.033928 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9a9e69-233c-46ff-9144-eef1f70445a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:16:57.034951 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9a9e69-233c-46ff-9144-eef1f70445a9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:16:57.034951 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9a9e69-233c-46ff-9144-eef1f70445a9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 15:16:57.035441 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9a9e69-233c-46ff-9144-eef1f70445a9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:16:57.037308 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9a9e69-233c-46ff-9144-eef1f70445a9 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:16:57.092645 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1310/2604] 10.222.0.46 () {58 vars in 1152 bytes} [Thu May 26 15:16:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 15:16:57.097470 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f683267-f1ac-49f2-aba9-880db3025c6a None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:16:57.109992 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f683267-f1ac-49f2-aba9-880db3025c6a None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:16:57.140101 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8cc662-1170-47ac-bfa1-3c213dce66c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:16:57.141066 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8cc662-1170-47ac-bfa1-3c213dce66c3 admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:16:57.141503 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8cc662-1170-47ac-bfa1-3c213dce66c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:16:57.142065 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8cc662-1170-47ac-bfa1-3c213dce66c3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:16:57.143021 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8cc662-1170-47ac-bfa1-3c213dce66c3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:16:57.145402 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8cc662-1170-47ac-bfa1-3c213dce66c3 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:16:57.191431 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1295/2605] 10.222.0.46 () {60 vars in 1152 bytes} [Thu May 26 15:16:57 2022] POST /identity/v3/projects => generated 377 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 15:16:57.196179 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9a9e69-233c-46ff-9144-eef1f70445a9 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:16:57.212157 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9a9e69-233c-46ff-9144-eef1f70445a9 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:16:57.369017 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2e4cbe-fa5f-4c91-9036-8285b3c9fa43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:16:57.369958 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2e4cbe-fa5f-4c91-9036-8285b3c9fa43 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:16:57.370261 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2e4cbe-fa5f-4c91-9036-8285b3c9fa43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:16:57.370532 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a2e4cbe-fa5f-4c91-9036-8285b3c9fa43 admin admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:16:57.371270 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2e4cbe-fa5f-4c91-9036-8285b3c9fa43 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:16:57.375277 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a2e4cbe-fa5f-4c91-9036-8285b3c9fa43 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:16:57.423666 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1311/2606] 10.222.0.46 () {60 vars in 1143 bytes} [Thu May 26 15:16:57 2022] POST /identity/v3/users => generated 289 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 15:16:57.429467 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8cc662-1170-47ac-bfa1-3c213dce66c3 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:16:57.452110 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8cc662-1170-47ac-bfa1-3c213dce66c3 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:16:57.470074 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b31ad2c-b64a-4a9f-bfbd-6a6ff0befb7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:16:57.472116 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b31ad2c-b64a-4a9f-bfbd-6a6ff0befb7a admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:16:57.473956 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b31ad2c-b64a-4a9f-bfbd-6a6ff0befb7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:16:57.475236 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b31ad2c-b64a-4a9f-bfbd-6a6ff0befb7a admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:16:57.477038 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b31ad2c-b64a-4a9f-bfbd-6a6ff0befb7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:16:57.479675 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b31ad2c-b64a-4a9f-bfbd-6a6ff0befb7a admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:16:57.522158 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1296/2607] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 15:16:57 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 15:16:57.527603 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2e4cbe-fa5f-4c91-9036-8285b3c9fa43 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:16:57.545960 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a2e4cbe-fa5f-4c91-9036-8285b3c9fa43 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:16:57.580791 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf314a2-f264-49a9-b016-5c43cd2a63bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:16:57.581798 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf314a2-f264-49a9-b016-5c43cd2a63bc admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:16:57.582096 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf314a2-f264-49a9-b016-5c43cd2a63bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:16:57.582384 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf314a2-f264-49a9-b016-5c43cd2a63bc admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:16:57.583149 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf314a2-f264-49a9-b016-5c43cd2a63bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:16:57.585578 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf314a2-f264-49a9-b016-5c43cd2a63bc admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:16:57.597880 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1312/2608] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 15:16:57 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 15:16:57.615437 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b31ad2c-b64a-4a9f-bfbd-6a6ff0befb7a admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:16:57.676228 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b31ad2c-b64a-4a9f-bfbd-6a6ff0befb7a admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:16:57.715047 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c3fd56-8ef0-424f-86bc-0e0d01ecf073 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:16:57.716715 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c3fd56-8ef0-424f-86bc-0e0d01ecf073 admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:16:57.717195 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c3fd56-8ef0-424f-86bc-0e0d01ecf073 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:16:57.717717 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c3fd56-8ef0-424f-86bc-0e0d01ecf073 admin admin] PATH_INFO: `/v3/projects/0d208300068a473cbdd7d44c78a8ede1/users/634f2acab57d40e5894a8f5646053f76/roles/d4088d1845a24fbd9bd8975ce0b1d3d0` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:16:57.764700 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c3fd56-8ef0-424f-86bc-0e0d01ecf073 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0d208300068a473cbdd7d44c78a8ede1, user_id=634f2acab57d40e5894a8f5646053f76, role_id=d4088d1845a24fbd9bd8975ce0b1d3d0)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:16:57.772953 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c3fd56-8ef0-424f-86bc-0e0d01ecf073 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:16:57.865724 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1297/2609] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 15:16:57 2022] PUT /identity/v3/projects/0d208300068a473cbdd7d44c78a8ede1/users/634f2acab57d40e5894a8f5646053f76/roles/d4088d1845a24fbd9bd8975ce0b1d3d0 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:16:57.873500 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df6b37c-907f-4167-8569-978ef31b4632 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:16:57.874065 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df6b37c-907f-4167-8569-978ef31b4632 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:16:57.874065 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df6b37c-907f-4167-8569-978ef31b4632 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:16:57.881674 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:16:57.928493 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-3df6b37c-907f-4167-8569-978ef31b4632 None None] MFA Rules not processed for user `634f2acab57d40e5894a8f5646053f76`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:16:57.929909 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-3df6b37c-907f-4167-8569-978ef31b4632 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:16:58.056289 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1313/2610] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:16:57 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:16:58.061784 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c3fd56-8ef0-424f-86bc-0e0d01ecf073 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:16:58.195824 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c3fd56-8ef0-424f-86bc-0e0d01ecf073 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:16:58.202324 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0f2a3d-5990-457a-ad2a-49a51a741cd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:16:58.202717 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0f2a3d-5990-457a-ad2a-49a51a741cd5 admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:16:58.202970 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0f2a3d-5990-457a-ad2a-49a51a741cd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:16:58.203211 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0f2a3d-5990-457a-ad2a-49a51a741cd5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:16:58.203907 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0f2a3d-5990-457a-ad2a-49a51a741cd5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:16:58.207398 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b0f2a3d-5990-457a-ad2a-49a51a741cd5 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:16:58.259010 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1298/2611] 10.222.0.46 () {60 vars in 1152 bytes} [Thu May 26 15:16:58 2022] POST /identity/v3/projects => generated 377 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 15:16:58.262625 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df6b37c-907f-4167-8569-978ef31b4632 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:16:58.284241 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df6b37c-907f-4167-8569-978ef31b4632 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:16:58.390280 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9be368d-d1ab-4074-812f-7fa22f2c5db9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:16:58.391642 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9be368d-d1ab-4074-812f-7fa22f2c5db9 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:16:58.392145 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9be368d-d1ab-4074-812f-7fa22f2c5db9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:16:58.392633 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9be368d-d1ab-4074-812f-7fa22f2c5db9 admin admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:16:58.393816 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9be368d-d1ab-4074-812f-7fa22f2c5db9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:16:58.396277 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9be368d-d1ab-4074-812f-7fa22f2c5db9 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:16:58.430513 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1314/2612] 10.222.0.46 () {60 vars in 1143 bytes} [Thu May 26 15:16:58 2022] POST /identity/v3/users => generated 289 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 15:16:58.431782 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0f2a3d-5990-457a-ad2a-49a51a741cd5 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:16:58.442350 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0f2a3d-5990-457a-ad2a-49a51a741cd5 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:16:58.447572 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a716fea4-102b-4575-b17d-61fb67188e3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:16:58.448390 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a716fea4-102b-4575-b17d-61fb67188e3d admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:16:58.448692 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a716fea4-102b-4575-b17d-61fb67188e3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:16:58.449018 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a716fea4-102b-4575-b17d-61fb67188e3d admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:16:58.449937 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a716fea4-102b-4575-b17d-61fb67188e3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:16:58.451797 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a716fea4-102b-4575-b17d-61fb67188e3d admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:16:58.480657 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1299/2613] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 15:16:58 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 15:16:58.486088 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9be368d-d1ab-4074-812f-7fa22f2c5db9 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:16:58.497820 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9be368d-d1ab-4074-812f-7fa22f2c5db9 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:16:58.516899 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5557e460-4313-4ad4-98d7-d5d57643c73a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:16:58.516899 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5557e460-4313-4ad4-98d7-d5d57643c73a admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:16:58.516899 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5557e460-4313-4ad4-98d7-d5d57643c73a admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:16:58.516899 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5557e460-4313-4ad4-98d7-d5d57643c73a admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:16:58.517738 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5557e460-4313-4ad4-98d7-d5d57643c73a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:16:58.517738 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5557e460-4313-4ad4-98d7-d5d57643c73a admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:16:58.523910 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1315/2614] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 15:16:58 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 15:16:58.528855 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a716fea4-102b-4575-b17d-61fb67188e3d admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:16:58.547589 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a716fea4-102b-4575-b17d-61fb67188e3d admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:16:58.553229 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32dfe323-d039-4fe4-91d7-8eb91ddb9156 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:16:58.554452 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dfe323-d039-4fe4-91d7-8eb91ddb9156 admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:16:58.555420 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dfe323-d039-4fe4-91d7-8eb91ddb9156 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:16:58.555874 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dfe323-d039-4fe4-91d7-8eb91ddb9156 admin admin] PATH_INFO: `/v3/projects/2af484362238496d916c011846561288/users/ebf75019186e4b93b821b5947a705fb4/roles/d4088d1845a24fbd9bd8975ce0b1d3d0` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:16:58.558898 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:16:58.580355 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32dfe323-d039-4fe4-91d7-8eb91ddb9156 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2af484362238496d916c011846561288, user_id=ebf75019186e4b93b821b5947a705fb4, role_id=d4088d1845a24fbd9bd8975ce0b1d3d0)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:16:58.588618 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32dfe323-d039-4fe4-91d7-8eb91ddb9156 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:16:58.607612 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1300/2615] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 15:16:58 2022] PUT /identity/v3/projects/2af484362238496d916c011846561288/users/ebf75019186e4b93b821b5947a705fb4/roles/d4088d1845a24fbd9bd8975ce0b1d3d0 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:16:58.620175 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d58871c-f553-4427-9439-8dd3658ab717 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:16:58.620503 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d58871c-f553-4427-9439-8dd3658ab717 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:16:58.620739 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d58871c-f553-4427-9439-8dd3658ab717 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:16:58.670904 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-5d58871c-f553-4427-9439-8dd3658ab717 None None] MFA Rules not processed for user `ebf75019186e4b93b821b5947a705fb4`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:16:58.672741 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-5d58871c-f553-4427-9439-8dd3658ab717 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:16:58.780284 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1316/2616] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:16:58 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:16:58.785484 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32dfe323-d039-4fe4-91d7-8eb91ddb9156 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:16:58.914765 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32dfe323-d039-4fe4-91d7-8eb91ddb9156 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:16:58.922527 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63b73b2-09c2-47f5-be90-068fce33255d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:16:58.922527 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63b73b2-09c2-47f5-be90-068fce33255d admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:16:58.922527 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63b73b2-09c2-47f5-be90-068fce33255d admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:16:58.924872 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63b73b2-09c2-47f5-be90-068fce33255d admin admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:16:58.924872 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63b73b2-09c2-47f5-be90-068fce33255d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:16:58.926345 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63b73b2-09c2-47f5-be90-068fce33255d admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:16:58.968666 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d58871c-f553-4427-9439-8dd3658ab717 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:16:58.975189 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1301/2617] 10.222.0.46 () {60 vars in 1152 bytes} [Thu May 26 15:16:58 2022] POST /identity/v3/projects => generated 379 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 15:16:58.975582 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:16:58.987860 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d58871c-f553-4427-9439-8dd3658ab717 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:16:59.108404 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c959267-c15c-4c2b-be4a-71b8d49a0225 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:16:59.109100 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c959267-c15c-4c2b-be4a-71b8d49a0225 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:16:59.109384 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c959267-c15c-4c2b-be4a-71b8d49a0225 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:16:59.109674 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c959267-c15c-4c2b-be4a-71b8d49a0225 admin admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:16:59.110485 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c959267-c15c-4c2b-be4a-71b8d49a0225 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:16:59.112434 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c959267-c15c-4c2b-be4a-71b8d49a0225 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:16:59.162713 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63b73b2-09c2-47f5-be90-068fce33255d admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:16:59.171498 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1317/2618] 10.222.0.46 () {60 vars in 1143 bytes} [Thu May 26 15:16:58 2022] POST /identity/v3/users => generated 296 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 15:16:59.177922 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63b73b2-09c2-47f5-be90-068fce33255d admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:16:59.188402 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f9e087-61e8-4d9d-989e-a83f1f225f8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:16:59.189342 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f9e087-61e8-4d9d-989e-a83f1f225f8c admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:16:59.189707 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f9e087-61e8-4d9d-989e-a83f1f225f8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:16:59.190774 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9f9e087-61e8-4d9d-989e-a83f1f225f8c admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:16:59.191679 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f9e087-61e8-4d9d-989e-a83f1f225f8c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:16:59.195193 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9f9e087-61e8-4d9d-989e-a83f1f225f8c admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:16:59.219267 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1302/2619] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 15:16:59 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 15:16:59.224429 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c959267-c15c-4c2b-be4a-71b8d49a0225 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:16:59.242583 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c959267-c15c-4c2b-be4a-71b8d49a0225 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:16:59.262956 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49b336d-ab15-4d11-9ca3-4ecca0eb0fc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:16:59.267166 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49b336d-ab15-4d11-9ca3-4ecca0eb0fc6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:16:59.267166 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49b336d-ab15-4d11-9ca3-4ecca0eb0fc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:16:59.267638 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49b336d-ab15-4d11-9ca3-4ecca0eb0fc6 admin admin] PATH_INFO: `/v3/projects/b6baea0c1c654e328dc14e807ee19277/users/a624ba128a8640aa9458c7aefaf5a00d/roles/a2e551773cbd4df084020676865b8222` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:16:59.301809 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49b336d-ab15-4d11-9ca3-4ecca0eb0fc6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b6baea0c1c654e328dc14e807ee19277, user_id=a624ba128a8640aa9458c7aefaf5a00d, role_id=a2e551773cbd4df084020676865b8222)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:16:59.307159 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49b336d-ab15-4d11-9ca3-4ecca0eb0fc6 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:16:59.333682 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1318/2620] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 15:16:59 2022] PUT /identity/v3/projects/b6baea0c1c654e328dc14e807ee19277/users/a624ba128a8640aa9458c7aefaf5a00d/roles/a2e551773cbd4df084020676865b8222 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:16:59.338246 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f9e087-61e8-4d9d-989e-a83f1f225f8c admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:16:59.356031 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9f9e087-61e8-4d9d-989e-a83f1f225f8c admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:16:59.471339 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4eb2832-27dc-4bee-ab34-9601472a81a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:16:59.474260 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4eb2832-27dc-4bee-ab34-9601472a81a1 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:16:59.475149 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4eb2832-27dc-4bee-ab34-9601472a81a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:16:59.475943 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4eb2832-27dc-4bee-ab34-9601472a81a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:16:59.477723 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4eb2832-27dc-4bee-ab34-9601472a81a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:16:59.481187 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4eb2832-27dc-4bee-ab34-9601472a81a1 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:16:59.505856 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1303/2621] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 15:16:59 2022] GET /identity/v3/roles => generated 1420 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 15:16:59.511540 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49b336d-ab15-4d11-9ca3-4ecca0eb0fc6 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:16:59.527828 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49b336d-ab15-4d11-9ca3-4ecca0eb0fc6 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:16:59.533615 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad32708c-3653-4190-9ca0-16f8be55519c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:16:59.534484 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad32708c-3653-4190-9ca0-16f8be55519c admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:16:59.534769 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad32708c-3653-4190-9ca0-16f8be55519c admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:16:59.535117 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad32708c-3653-4190-9ca0-16f8be55519c admin admin] PATH_INFO: `/v3/projects/b6baea0c1c654e328dc14e807ee19277/users/a624ba128a8640aa9458c7aefaf5a00d/roles/a2e551773cbd4df084020676865b8222` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:16:59.537213 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad32708c-3653-4190-9ca0-16f8be55519c admin admin] RBAC: Authorizing `identity:create_grant(project_id=b6baea0c1c654e328dc14e807ee19277, user_id=a624ba128a8640aa9458c7aefaf5a00d, role_id=a2e551773cbd4df084020676865b8222)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:16:59.539946 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad32708c-3653-4190-9ca0-16f8be55519c admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:16:59.552217 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1319/2622] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 15:16:59 2022] PUT /identity/v3/projects/b6baea0c1c654e328dc14e807ee19277/users/a624ba128a8640aa9458c7aefaf5a00d/roles/a2e551773cbd4df084020676865b8222 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:16:59.567981 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377334a3-d6a7-4eb5-980a-b5a32b77b328 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:16:59.568354 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377334a3-d6a7-4eb5-980a-b5a32b77b328 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:16:59.568770 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377334a3-d6a7-4eb5-980a-b5a32b77b328 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:16:59.618873 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-377334a3-d6a7-4eb5-980a-b5a32b77b328 None None] MFA Rules not processed for user `a624ba128a8640aa9458c7aefaf5a00d`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:16:59.621547 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-377334a3-d6a7-4eb5-980a-b5a32b77b328 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:16:59.702829 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:16:59.756762 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1304/2623] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:16:59 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 15:16:59.769239 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad32708c-3653-4190-9ca0-16f8be55519c admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:16:59.771202 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-ad32708c-3653-4190-9ca0-16f8be55519c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:16:59.885439 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad32708c-3653-4190-9ca0-16f8be55519c admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:16:59.889783 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac35a89-8aa5-4ed4-b8ae-9e7223d7176e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5b5b35614a547d783e6e77ddb8c1c03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:16:59.890869 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac35a89-8aa5-4ed4-b8ae-9e7223d7176e service cinder] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:16:59.891318 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac35a89-8aa5-4ed4-b8ae-9e7223d7176e service cinder] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:16:59.891737 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac35a89-8aa5-4ed4-b8ae-9e7223d7176e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:16:59.909859 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac35a89-8aa5-4ed4-b8ae-9e7223d7176e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:16:59.912841 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac35a89-8aa5-4ed4-b8ae-9e7223d7176e service cinder] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:16:59.935548 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1320/2624] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:16:59 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 15:17:00.012435 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377334a3-d6a7-4eb5-980a-b5a32b77b328 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:17:00.013943 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-377334a3-d6a7-4eb5-980a-b5a32b77b328 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:17:00.027402 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377334a3-d6a7-4eb5-980a-b5a32b77b328 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:17:00.031567 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9320a5-dca9-4145-8168-c04846d7c129 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5b5b35614a547d783e6e77ddb8c1c03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:17:00.032336 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9320a5-dca9-4145-8168-c04846d7c129 service cinder] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:17:00.032705 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9320a5-dca9-4145-8168-c04846d7c129 service cinder] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:17:00.032990 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9320a5-dca9-4145-8168-c04846d7c129 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:17:00.132174 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9320a5-dca9-4145-8168-c04846d7c129 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:17:00.133912 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9320a5-dca9-4145-8168-c04846d7c129 service cinder] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:17:00.158758 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1305/2625] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:17:00 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 15:17:05.285464 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac35a89-8aa5-4ed4-b8ae-9e7223d7176e service cinder] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:17:05.354046 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac35a89-8aa5-4ed4-b8ae-9e7223d7176e service cinder] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:17:05.357719 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:17:05.359364 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d648c3-3e16-460f-afde-5b349590ad3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:17:05.361238 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d648c3-3e16-460f-afde-5b349590ad3a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:17:05.361588 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d648c3-3e16-460f-afde-5b349590ad3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:17:05.361911 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d648c3-3e16-460f-afde-5b349590ad3a admin admin] PATH_INFO: `/v3/users/634f2acab57d40e5894a8f5646053f76` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:17:05.363326 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d648c3-3e16-460f-afde-5b349590ad3a admin admin] RBAC: Authorizing `identity:delete_user(user_id=634f2acab57d40e5894a8f5646053f76)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:17:05.366695 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d648c3-3e16-460f-afde-5b349590ad3a admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:17:05.421560 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-d2d648c3-3e16-460f-afde-5b349590ad3a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '634f2acab57d40e5894a8f5646053f76'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:17:05.431590 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-d2d648c3-3e16-460f-afde-5b349590ad3a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '634f2acab57d40e5894a8f5646053f76'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:17:05.432345 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-d2d648c3-3e16-460f-afde-5b349590ad3a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '634f2acab57d40e5894a8f5646053f76'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:17:05.449782 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-d2d648c3-3e16-460f-afde-5b349590ad3a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '634f2acab57d40e5894a8f5646053f76'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:17:05.457684 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-d2d648c3-3e16-460f-afde-5b349590ad3a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '634f2acab57d40e5894a8f5646053f76'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:17:05.475474 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1321/2626] 10.222.0.46 () {58 vars in 1223 bytes} [Thu May 26 15:17:05 2022] DELETE /identity/v3/users/634f2acab57d40e5894a8f5646053f76 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:17:05.481117 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d991e1bf-bf90-4a13-9d82-45654ade912d None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:17:05.482739 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d991e1bf-bf90-4a13-9d82-45654ade912d None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:17:05.483699 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d991e1bf-bf90-4a13-9d82-45654ade912d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:17:05.507820 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-d991e1bf-bf90-4a13-9d82-45654ade912d None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:17:05.509118 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-d991e1bf-bf90-4a13-9d82-45654ade912d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:17:05.610040 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1306/2627] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:17:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:17:05.618455 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d648c3-3e16-460f-afde-5b349590ad3a admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:17:05.626964 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-d2d648c3-3e16-460f-afde-5b349590ad3a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:17:05.698859 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d648c3-3e16-460f-afde-5b349590ad3a admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:17:05.702559 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b91f43-bb16-4e42-adb4-b7bccc305b25 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0156a2f6f3794f479e76b85bdbbe5c26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:17:05.703178 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b91f43-bb16-4e42-adb4-b7bccc305b25 service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:17:05.703460 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b91f43-bb16-4e42-adb4-b7bccc305b25 service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:17:05.703737 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b91f43-bb16-4e42-adb4-b7bccc305b25 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:17:05.713366 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b91f43-bb16-4e42-adb4-b7bccc305b25 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:17:05.715452 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b91f43-bb16-4e42-adb4-b7bccc305b25 service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:17:05.724970 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1322/2628] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:17:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 15:17:06.006915 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d991e1bf-bf90-4a13-9d82-45654ade912d None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:17:06.009002 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-d991e1bf-bf90-4a13-9d82-45654ade912d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:17:06.024436 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d991e1bf-bf90-4a13-9d82-45654ade912d None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:17:06.029538 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336c716c-a673-44bc-b5c2-09cedcb6d56f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:17:06.030415 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336c716c-a673-44bc-b5c2-09cedcb6d56f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:17:06.030690 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336c716c-a673-44bc-b5c2-09cedcb6d56f admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:17:06.030970 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336c716c-a673-44bc-b5c2-09cedcb6d56f admin admin] PATH_INFO: `/v3/projects/0d208300068a473cbdd7d44c78a8ede1` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:17:06.032157 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336c716c-a673-44bc-b5c2-09cedcb6d56f admin admin] RBAC: Authorizing `identity:delete_project(project_id=0d208300068a473cbdd7d44c78a8ede1)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:17:06.034143 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336c716c-a673-44bc-b5c2-09cedcb6d56f admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:17:06.082480 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-336c716c-a673-44bc-b5c2-09cedcb6d56f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0d208300068a473cbdd7d44c78a8ede1', 'request_id': 'req-336c716c-a673-44bc-b5c2-09cedcb6d56f'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:17:06.090274 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-336c716c-a673-44bc-b5c2-09cedcb6d56f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0d208300068a473cbdd7d44c78a8ede1', 'request_id': 'req-336c716c-a673-44bc-b5c2-09cedcb6d56f'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:17:06.096380 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-336c716c-a673-44bc-b5c2-09cedcb6d56f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0d208300068a473cbdd7d44c78a8ede1', 'request_id': 'req-336c716c-a673-44bc-b5c2-09cedcb6d56f'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:17:06.136979 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:17:06.141925 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-336c716c-a673-44bc-b5c2-09cedcb6d56f admin admin] The token cache is being invalidate because project 0d208300068a473cbdd7d44c78a8ede1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 15:17:06.142452 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-336c716c-a673-44bc-b5c2-09cedcb6d56f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:17:06.143221 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-336c716c-a673-44bc-b5c2-09cedcb6d56f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:17:06.144987 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1307/2629] 10.222.0.46 () {58 vars in 1232 bytes} [Thu May 26 15:17:06 2022] DELETE /identity/v3/projects/0d208300068a473cbdd7d44c78a8ede1 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:17:06.155083 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b91f43-bb16-4e42-adb4-b7bccc305b25 service neutron] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:17:06.175348 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b91f43-bb16-4e42-adb4-b7bccc305b25 service neutron] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:17:06.265495 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5db94ca-3af9-456f-ae59-bfc4512ca2f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:17:06.266369 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5db94ca-3af9-456f-ae59-bfc4512ca2f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:17:06.266648 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5db94ca-3af9-456f-ae59-bfc4512ca2f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:17:06.266923 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5db94ca-3af9-456f-ae59-bfc4512ca2f9 admin admin] PATH_INFO: `/v3/users/ebf75019186e4b93b821b5947a705fb4` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:17:06.268097 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5db94ca-3af9-456f-ae59-bfc4512ca2f9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ebf75019186e4b93b821b5947a705fb4)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:17:06.270081 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5db94ca-3af9-456f-ae59-bfc4512ca2f9 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:17:06.295292 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:17:06.328506 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-e5db94ca-3af9-456f-ae59-bfc4512ca2f9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ebf75019186e4b93b821b5947a705fb4'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:17:06.337506 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-e5db94ca-3af9-456f-ae59-bfc4512ca2f9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ebf75019186e4b93b821b5947a705fb4'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:17:06.338346 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-e5db94ca-3af9-456f-ae59-bfc4512ca2f9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ebf75019186e4b93b821b5947a705fb4'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:17:06.355696 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-e5db94ca-3af9-456f-ae59-bfc4512ca2f9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ebf75019186e4b93b821b5947a705fb4'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:17:06.370439 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-e5db94ca-3af9-456f-ae59-bfc4512ca2f9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ebf75019186e4b93b821b5947a705fb4'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:17:06.377524 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1323/2630] 10.222.0.46 () {58 vars in 1223 bytes} [Thu May 26 15:17:06 2022] DELETE /identity/v3/users/ebf75019186e4b93b821b5947a705fb4 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:17:06.625309 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336c716c-a673-44bc-b5c2-09cedcb6d56f admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:17:06.627453 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-336c716c-a673-44bc-b5c2-09cedcb6d56f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:17:06.641652 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336c716c-a673-44bc-b5c2-09cedcb6d56f admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:17:06.686406 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5db94ca-3af9-456f-ae59-bfc4512ca2f9 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:17:06.689117 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-e5db94ca-3af9-456f-ae59-bfc4512ca2f9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:17:06.734253 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945e8535-bf80-4d26-a3c7-12fe3c408c75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:17:06.735783 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945e8535-bf80-4d26-a3c7-12fe3c408c75 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:17:06.736079 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945e8535-bf80-4d26-a3c7-12fe3c408c75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:17:06.736376 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945e8535-bf80-4d26-a3c7-12fe3c408c75 admin admin] PATH_INFO: `/v3/users/da62bffb283b4fb8b4c01083e8c7cb87` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:17:06.737864 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945e8535-bf80-4d26-a3c7-12fe3c408c75 admin admin] RBAC: Authorizing `identity:delete_user(user_id=da62bffb283b4fb8b4c01083e8c7cb87)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:17:06.740499 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945e8535-bf80-4d26-a3c7-12fe3c408c75 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:17:06.797641 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5db94ca-3af9-456f-ae59-bfc4512ca2f9 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:17:06.802780 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-945e8535-bf80-4d26-a3c7-12fe3c408c75 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'da62bffb283b4fb8b4c01083e8c7cb87'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:17:06.803018 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447937af-b5ed-41b8-a4be-93c13ce4ee04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:17:06.803831 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447937af-b5ed-41b8-a4be-93c13ce4ee04 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:17:06.804117 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447937af-b5ed-41b8-a4be-93c13ce4ee04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:17:06.804421 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447937af-b5ed-41b8-a4be-93c13ce4ee04 admin admin] PATH_INFO: `/v3/projects/2af484362238496d916c011846561288` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:17:06.805684 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447937af-b5ed-41b8-a4be-93c13ce4ee04 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2af484362238496d916c011846561288)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:17:06.807903 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447937af-b5ed-41b8-a4be-93c13ce4ee04 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:17:06.812279 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-945e8535-bf80-4d26-a3c7-12fe3c408c75 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'da62bffb283b4fb8b4c01083e8c7cb87'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:17:06.813042 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-945e8535-bf80-4d26-a3c7-12fe3c408c75 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'da62bffb283b4fb8b4c01083e8c7cb87'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:17:06.829006 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-945e8535-bf80-4d26-a3c7-12fe3c408c75 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'da62bffb283b4fb8b4c01083e8c7cb87'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:17:06.839090 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-945e8535-bf80-4d26-a3c7-12fe3c408c75 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'da62bffb283b4fb8b4c01083e8c7cb87'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:17:06.847070 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1308/2631] 10.222.0.46 () {58 vars in 1223 bytes} [Thu May 26 15:17:06 2022] DELETE /identity/v3/users/da62bffb283b4fb8b4c01083e8c7cb87 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:17:06.851726 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d3a070-e31d-4800-89a0-996cef4bfcd8 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:17:06.852017 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d3a070-e31d-4800-89a0-996cef4bfcd8 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:17:06.852297 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d3a070-e31d-4800-89a0-996cef4bfcd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:17:06.870438 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-91d3a070-e31d-4800-89a0-996cef4bfcd8 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:17:06.871662 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-91d3a070-e31d-4800-89a0-996cef4bfcd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:17:06.879189 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-447937af-b5ed-41b8-a4be-93c13ce4ee04 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2af484362238496d916c011846561288', 'request_id': 'req-447937af-b5ed-41b8-a4be-93c13ce4ee04'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:17:06.889553 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-447937af-b5ed-41b8-a4be-93c13ce4ee04 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2af484362238496d916c011846561288', 'request_id': 'req-447937af-b5ed-41b8-a4be-93c13ce4ee04'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:17:06.899225 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-447937af-b5ed-41b8-a4be-93c13ce4ee04 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2af484362238496d916c011846561288', 'request_id': 'req-447937af-b5ed-41b8-a4be-93c13ce4ee04'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:17:06.912280 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-447937af-b5ed-41b8-a4be-93c13ce4ee04 admin admin] The token cache is being invalidate because project 2af484362238496d916c011846561288 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 15:17:06.912569 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-447937af-b5ed-41b8-a4be-93c13ce4ee04 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:17:06.913284 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-447937af-b5ed-41b8-a4be-93c13ce4ee04 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:17:06.914880 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1324/2632] 10.222.0.46 () {58 vars in 1232 bytes} [Thu May 26 15:17:06 2022] DELETE /identity/v3/projects/2af484362238496d916c011846561288 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:17:06.919086 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447937af-b5ed-41b8-a4be-93c13ce4ee04 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:17:06.920522 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-447937af-b5ed-41b8-a4be-93c13ce4ee04 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:17:06.938196 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1309/2633] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:17:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:17:06.948465 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d3a070-e31d-4800-89a0-996cef4bfcd8 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:17:06.949876 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-91d3a070-e31d-4800-89a0-996cef4bfcd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:17:06.954996 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:17:06.973929 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:17:06.992376 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447937af-b5ed-41b8-a4be-93c13ce4ee04 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:17:06.997180 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81650120-b448-4e4e-a2be-068d5dd11f4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:17:06.998093 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81650120-b448-4e4e-a2be-068d5dd11f4f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:17:06.998385 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81650120-b448-4e4e-a2be-068d5dd11f4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:17:06.998663 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81650120-b448-4e4e-a2be-068d5dd11f4f admin admin] PATH_INFO: `/v3/users/a624ba128a8640aa9458c7aefaf5a00d` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:17:06.999871 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81650120-b448-4e4e-a2be-068d5dd11f4f admin admin] RBAC: Authorizing `identity:delete_user(user_id=a624ba128a8640aa9458c7aefaf5a00d)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:17:07.001843 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81650120-b448-4e4e-a2be-068d5dd11f4f admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:17:07.013926 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d3a070-e31d-4800-89a0-996cef4bfcd8 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:17:07.017599 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74ace04-21c9-431b-acdd-bbaef5cd940d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0156a2f6f3794f479e76b85bdbbe5c26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:17:07.018249 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74ace04-21c9-431b-acdd-bbaef5cd940d service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:17:07.018537 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74ace04-21c9-431b-acdd-bbaef5cd940d service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:17:07.018815 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74ace04-21c9-431b-acdd-bbaef5cd940d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:17:07.029862 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74ace04-21c9-431b-acdd-bbaef5cd940d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:17:07.031320 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74ace04-21c9-431b-acdd-bbaef5cd940d service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:17:07.042146 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1310/2634] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:17:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 15:17:07.051138 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-81650120-b448-4e4e-a2be-068d5dd11f4f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a624ba128a8640aa9458c7aefaf5a00d'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:17:07.060449 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-81650120-b448-4e4e-a2be-068d5dd11f4f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a624ba128a8640aa9458c7aefaf5a00d'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:17:07.061175 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-81650120-b448-4e4e-a2be-068d5dd11f4f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a624ba128a8640aa9458c7aefaf5a00d'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:17:07.074958 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-81650120-b448-4e4e-a2be-068d5dd11f4f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a624ba128a8640aa9458c7aefaf5a00d'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:17:07.082943 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-81650120-b448-4e4e-a2be-068d5dd11f4f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a624ba128a8640aa9458c7aefaf5a00d'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:17:07.090185 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1325/2635] 10.222.0.46 () {58 vars in 1223 bytes} [Thu May 26 15:17:06 2022] DELETE /identity/v3/users/a624ba128a8640aa9458c7aefaf5a00d => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 15:17:07.335979 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74ace04-21c9-431b-acdd-bbaef5cd940d service neutron] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:17:07.337850 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-a74ace04-21c9-431b-acdd-bbaef5cd940d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:17:07.364611 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74ace04-21c9-431b-acdd-bbaef5cd940d service neutron] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:17:07.409373 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81650120-b448-4e4e-a2be-068d5dd11f4f admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:17:07.410938 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-81650120-b448-4e4e-a2be-068d5dd11f4f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:17:07.460396 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08de363-87e6-46a4-bf52-35dd89118f0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:17:07.460396 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08de363-87e6-46a4-bf52-35dd89118f0a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:17:07.460832 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08de363-87e6-46a4-bf52-35dd89118f0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:17:07.461200 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08de363-87e6-46a4-bf52-35dd89118f0a admin admin] PATH_INFO: `/v3/projects/d639caecc6dc49558e374eb624a0fe91` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:17:07.462602 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08de363-87e6-46a4-bf52-35dd89118f0a admin admin] RBAC: Authorizing `identity:delete_project(project_id=d639caecc6dc49558e374eb624a0fe91)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:17:07.464776 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08de363-87e6-46a4-bf52-35dd89118f0a admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:17:07.498445 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81650120-b448-4e4e-a2be-068d5dd11f4f admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:17:07.505788 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b178337d-14cd-4ff0-b8a2-97dbbc78279e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:17:07.506983 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b178337d-14cd-4ff0-b8a2-97dbbc78279e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:17:07.507495 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b178337d-14cd-4ff0-b8a2-97dbbc78279e admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:17:07.507841 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b178337d-14cd-4ff0-b8a2-97dbbc78279e admin admin] PATH_INFO: `/v3/projects/b6baea0c1c654e328dc14e807ee19277` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:17:07.509556 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b178337d-14cd-4ff0-b8a2-97dbbc78279e admin admin] RBAC: Authorizing `identity:delete_project(project_id=b6baea0c1c654e328dc14e807ee19277)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:17:07.511797 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b178337d-14cd-4ff0-b8a2-97dbbc78279e admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:17:07.533488 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-f08de363-87e6-46a4-bf52-35dd89118f0a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd639caecc6dc49558e374eb624a0fe91', 'request_id': 'req-f08de363-87e6-46a4-bf52-35dd89118f0a'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:17:07.543100 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-f08de363-87e6-46a4-bf52-35dd89118f0a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd639caecc6dc49558e374eb624a0fe91', 'request_id': 'req-f08de363-87e6-46a4-bf52-35dd89118f0a'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:17:07.554227 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-f08de363-87e6-46a4-bf52-35dd89118f0a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd639caecc6dc49558e374eb624a0fe91', 'request_id': 'req-f08de363-87e6-46a4-bf52-35dd89118f0a'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:17:07.573066 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-f08de363-87e6-46a4-bf52-35dd89118f0a admin admin] The token cache is being invalidate because project d639caecc6dc49558e374eb624a0fe91 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 15:17:07.573760 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-f08de363-87e6-46a4-bf52-35dd89118f0a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:17:07.573896 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-b178337d-14cd-4ff0-b8a2-97dbbc78279e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b6baea0c1c654e328dc14e807ee19277', 'request_id': 'req-b178337d-14cd-4ff0-b8a2-97dbbc78279e'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:17:07.574996 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-f08de363-87e6-46a4-bf52-35dd89118f0a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:17:07.576946 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1311/2636] 10.222.0.46 () {58 vars in 1232 bytes} [Thu May 26 15:17:07 2022] DELETE /identity/v3/projects/d639caecc6dc49558e374eb624a0fe91 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:17:07.578309 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:17:07.580904 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08de363-87e6-46a4-bf52-35dd89118f0a admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:17:07.582427 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-f08de363-87e6-46a4-bf52-35dd89118f0a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:17:07.583778 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-b178337d-14cd-4ff0-b8a2-97dbbc78279e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b6baea0c1c654e328dc14e807ee19277', 'request_id': 'req-b178337d-14cd-4ff0-b8a2-97dbbc78279e'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:17:07.592758 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-b178337d-14cd-4ff0-b8a2-97dbbc78279e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b6baea0c1c654e328dc14e807ee19277', 'request_id': 'req-b178337d-14cd-4ff0-b8a2-97dbbc78279e'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:17:07.606653 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-b178337d-14cd-4ff0-b8a2-97dbbc78279e admin admin] The token cache is being invalidate because project b6baea0c1c654e328dc14e807ee19277 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 15:17:07.608253 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-b178337d-14cd-4ff0-b8a2-97dbbc78279e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:17:07.608253 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-b178337d-14cd-4ff0-b8a2-97dbbc78279e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:17:07.609582 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1326/2637] 10.222.0.46 () {58 vars in 1232 bytes} [Thu May 26 15:17:07 2022] DELETE /identity/v3/projects/b6baea0c1c654e328dc14e807ee19277 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:17:07.616138 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa87097-9b38-4891-837d-0ae719c566cb None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:17:07.616640 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa87097-9b38-4891-837d-0ae719c566cb None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:17:07.617572 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa87097-9b38-4891-837d-0ae719c566cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:17:07.638717 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-caa87097-9b38-4891-837d-0ae719c566cb None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:17:07.640530 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-caa87097-9b38-4891-837d-0ae719c566cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:17:07.686958 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:17:07.706059 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08de363-87e6-46a4-bf52-35dd89118f0a admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:17:07.716433 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1327/2638] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:17:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:17:07.724850 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05aea600-bf5a-490f-8b39-9cc8a4cc16d4 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:17:07.725140 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05aea600-bf5a-490f-8b39-9cc8a4cc16d4 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:17:07.725423 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05aea600-bf5a-490f-8b39-9cc8a4cc16d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:17:07.750561 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-05aea600-bf5a-490f-8b39-9cc8a4cc16d4 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:17:07.751853 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-05aea600-bf5a-490f-8b39-9cc8a4cc16d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:17:07.760464 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1328/2639] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:17:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:17:07.765418 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05aea600-bf5a-490f-8b39-9cc8a4cc16d4 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:17:07.774600 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a40cd0-c1fd-4e4b-947a-8af8f1ee3723 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:17:07.775815 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a40cd0-c1fd-4e4b-947a-8af8f1ee3723 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:17:07.776338 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a40cd0-c1fd-4e4b-947a-8af8f1ee3723 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:17:07.776548 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a40cd0-c1fd-4e4b-947a-8af8f1ee3723 admin admin] PATH_INFO: `/v3/users/19b99c92a79b47c388f687f21de2c1cf` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:17:07.777017 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05aea600-bf5a-490f-8b39-9cc8a4cc16d4 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:17:07.778086 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a40cd0-c1fd-4e4b-947a-8af8f1ee3723 admin admin] RBAC: Authorizing `identity:delete_user(user_id=19b99c92a79b47c388f687f21de2c1cf)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:17:07.780168 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a40cd0-c1fd-4e4b-947a-8af8f1ee3723 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:17:07.782422 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a283657-b474-439d-a983-ac6f86f10687 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:17:07.783956 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a283657-b474-439d-a983-ac6f86f10687 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:17:07.784203 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a283657-b474-439d-a983-ac6f86f10687 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:17:07.784485 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a283657-b474-439d-a983-ac6f86f10687 admin admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:17:07.785490 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a283657-b474-439d-a983-ac6f86f10687 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 15:17:07.786013 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a283657-b474-439d-a983-ac6f86f10687 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:17:07.788552 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a283657-b474-439d-a983-ac6f86f10687 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:17:07.803290 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1329/2640] 10.222.0.46 () {58 vars in 1152 bytes} [Thu May 26 15:17:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 15:17:07.808507 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a283657-b474-439d-a983-ac6f86f10687 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:17:07.824665 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a283657-b474-439d-a983-ac6f86f10687 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:17:07.847266 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-59a40cd0-c1fd-4e4b-947a-8af8f1ee3723 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '19b99c92a79b47c388f687f21de2c1cf'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:17:07.850469 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfdc0a5-a0e3-4910-808c-a6cb395a7157 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:17:07.852775 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfdc0a5-a0e3-4910-808c-a6cb395a7157 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:17:07.853346 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfdc0a5-a0e3-4910-808c-a6cb395a7157 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:17:07.854231 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfdc0a5-a0e3-4910-808c-a6cb395a7157 admin admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:17:07.855519 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfdc0a5-a0e3-4910-808c-a6cb395a7157 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:17:07.858515 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfdc0a5-a0e3-4910-808c-a6cb395a7157 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:17:07.859622 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-59a40cd0-c1fd-4e4b-947a-8af8f1ee3723 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '19b99c92a79b47c388f687f21de2c1cf'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:17:07.860446 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-59a40cd0-c1fd-4e4b-947a-8af8f1ee3723 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '19b99c92a79b47c388f687f21de2c1cf'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:17:07.881623 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-59a40cd0-c1fd-4e4b-947a-8af8f1ee3723 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '19b99c92a79b47c388f687f21de2c1cf'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:17:07.892660 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-59a40cd0-c1fd-4e4b-947a-8af8f1ee3723 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '19b99c92a79b47c388f687f21de2c1cf'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:17:07.901616 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1330/2641] 10.222.0.46 () {60 vars in 1152 bytes} [Thu May 26 15:17:07 2022] POST /identity/v3/projects => generated 381 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 15:17:07.902438 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1312/2642] 10.222.0.46 () {58 vars in 1223 bytes} [Thu May 26 15:17:07 2022] DELETE /identity/v3/users/19b99c92a79b47c388f687f21de2c1cf => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:17:07.903046 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfdc0a5-a0e3-4910-808c-a6cb395a7157 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:17:07.913172 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-edfdc0a5-a0e3-4910-808c-a6cb395a7157 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:17:08.011852 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfdc0a5-a0e3-4910-808c-a6cb395a7157 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:17:08.068775 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:INSERT=2,keystone:DELETE=3,keystone:SELECT=95 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:17:08.103063 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375debfc-7556-48f2-8e0a-7986ad61fd13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:17:08.103777 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375debfc-7556-48f2-8e0a-7986ad61fd13 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:17:08.104057 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375debfc-7556-48f2-8e0a-7986ad61fd13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:17:08.104387 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375debfc-7556-48f2-8e0a-7986ad61fd13 admin admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:17:08.105134 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375debfc-7556-48f2-8e0a-7986ad61fd13 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:17:08.107228 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-375debfc-7556-48f2-8e0a-7986ad61fd13 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:17:08.136333 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1331/2643] 10.222.0.46 () {60 vars in 1143 bytes} [Thu May 26 15:17:07 2022] POST /identity/v3/users => generated 291 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 15:17:08.138674 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a40cd0-c1fd-4e4b-947a-8af8f1ee3723 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:17:08.157860 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a40cd0-c1fd-4e4b-947a-8af8f1ee3723 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:17:08.184667 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e3c6cd-6cf6-4654-acbc-c229402c84a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:17:08.186092 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e3c6cd-6cf6-4654-acbc-c229402c84a9 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:17:08.186804 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e3c6cd-6cf6-4654-acbc-c229402c84a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:17:08.187419 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e3c6cd-6cf6-4654-acbc-c229402c84a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:17:08.188719 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e3c6cd-6cf6-4654-acbc-c229402c84a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:17:08.191171 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e3c6cd-6cf6-4654-acbc-c229402c84a9 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:17:08.203457 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1313/2644] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 15:17:08 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 15:17:08.207425 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375debfc-7556-48f2-8e0a-7986ad61fd13 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:17:08.221502 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375debfc-7556-48f2-8e0a-7986ad61fd13 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:17:08.229046 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44441205-29be-466e-9887-437757535f6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:17:08.229790 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44441205-29be-466e-9887-437757535f6d admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:17:08.230075 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44441205-29be-466e-9887-437757535f6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:17:08.230355 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44441205-29be-466e-9887-437757535f6d admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:17:08.231093 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44441205-29be-466e-9887-437757535f6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:17:08.233175 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44441205-29be-466e-9887-437757535f6d admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:17:08.247589 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1332/2645] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 15:17:08 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 15:17:08.248784 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e3c6cd-6cf6-4654-acbc-c229402c84a9 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:17:08.259113 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e3c6cd-6cf6-4654-acbc-c229402c84a9 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:17:08.265626 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16473863-49af-4f9e-810c-2f18cc3e9dd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:17:08.266791 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16473863-49af-4f9e-810c-2f18cc3e9dd3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:17:08.267047 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16473863-49af-4f9e-810c-2f18cc3e9dd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:17:08.267414 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16473863-49af-4f9e-810c-2f18cc3e9dd3 admin admin] PATH_INFO: `/v3/projects/30f1cd47846941e0ab5e19d976a44d6c/users/49188f7b627d4be799e805e48496bf9c/roles/d4088d1845a24fbd9bd8975ce0b1d3d0` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:17:08.292307 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44441205-29be-466e-9887-437757535f6d admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:17:08.298085 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-44441205-29be-466e-9887-437757535f6d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:17:08.298576 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16473863-49af-4f9e-810c-2f18cc3e9dd3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=30f1cd47846941e0ab5e19d976a44d6c, user_id=49188f7b627d4be799e805e48496bf9c, role_id=d4088d1845a24fbd9bd8975ce0b1d3d0)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:17:08.301739 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16473863-49af-4f9e-810c-2f18cc3e9dd3 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:17:08.326680 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1314/2646] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 15:17:08 2022] PUT /identity/v3/projects/30f1cd47846941e0ab5e19d976a44d6c/users/49188f7b627d4be799e805e48496bf9c/roles/d4088d1845a24fbd9bd8975ce0b1d3d0 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:17:08.331336 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44441205-29be-466e-9887-437757535f6d admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:17:08.340147 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56886d5a-498e-4232-b0d7-a9096995694d None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:17:08.340914 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56886d5a-498e-4232-b0d7-a9096995694d None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:17:08.341276 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56886d5a-498e-4232-b0d7-a9096995694d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:17:08.374657 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-56886d5a-498e-4232-b0d7-a9096995694d None None] MFA Rules not processed for user `49188f7b627d4be799e805e48496bf9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:17:08.375924 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-56886d5a-498e-4232-b0d7-a9096995694d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:17:08.405877 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f7ab4d-320b-4c0e-a9b4-fdee50f4e857 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:17:08.406773 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f7ab4d-320b-4c0e-a9b4-fdee50f4e857 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:17:08.407066 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f7ab4d-320b-4c0e-a9b4-fdee50f4e857 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:17:08.407342 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f7ab4d-320b-4c0e-a9b4-fdee50f4e857 admin admin] PATH_INFO: `/v3/projects/29a409eaa55c4178bb5b41d7d6e37659` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:17:08.408562 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f7ab4d-320b-4c0e-a9b4-fdee50f4e857 admin admin] RBAC: Authorizing `identity:delete_project(project_id=29a409eaa55c4178bb5b41d7d6e37659)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:17:08.410525 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f7ab4d-320b-4c0e-a9b4-fdee50f4e857 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:17:08.412889 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:17:08.452056 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1315/2647] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:17:08 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 15:17:08.456496 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56886d5a-498e-4232-b0d7-a9096995694d None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:17:08.464921 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-79f7ab4d-320b-4c0e-a9b4-fdee50f4e857 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '29a409eaa55c4178bb5b41d7d6e37659', 'request_id': 'req-79f7ab4d-320b-4c0e-a9b4-fdee50f4e857'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:17:08.470593 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56886d5a-498e-4232-b0d7-a9096995694d None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:17:08.473918 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-79f7ab4d-320b-4c0e-a9b4-fdee50f4e857 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '29a409eaa55c4178bb5b41d7d6e37659', 'request_id': 'req-79f7ab4d-320b-4c0e-a9b4-fdee50f4e857'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:17:08.481755 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-79f7ab4d-320b-4c0e-a9b4-fdee50f4e857 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '29a409eaa55c4178bb5b41d7d6e37659', 'request_id': 'req-79f7ab4d-320b-4c0e-a9b4-fdee50f4e857'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:17:08.498036 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-79f7ab4d-320b-4c0e-a9b4-fdee50f4e857 admin admin] The token cache is being invalidate because project 29a409eaa55c4178bb5b41d7d6e37659 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 15:17:08.498036 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-79f7ab4d-320b-4c0e-a9b4-fdee50f4e857 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:17:08.498854 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-79f7ab4d-320b-4c0e-a9b4-fdee50f4e857 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:17:08.501035 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1333/2648] 10.222.0.46 () {58 vars in 1232 bytes} [Thu May 26 15:17:08 2022] DELETE /identity/v3/projects/29a409eaa55c4178bb5b41d7d6e37659 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:17:08.507953 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f315c6-8141-4fe5-a848-24915f30f408 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:17:08.508251 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f315c6-8141-4fe5-a848-24915f30f408 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:17:08.508530 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f315c6-8141-4fe5-a848-24915f30f408 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:17:08.529392 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-51f315c6-8141-4fe5-a848-24915f30f408 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:17:08.531085 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-51f315c6-8141-4fe5-a848-24915f30f408 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:17:08.535900 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e23c04-6334-4197-b2ad-ee96e7a21983 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:17:08.536576 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e23c04-6334-4197-b2ad-ee96e7a21983 admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:17:08.536989 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e23c04-6334-4197-b2ad-ee96e7a21983 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:17:08.537275 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e23c04-6334-4197-b2ad-ee96e7a21983 admin admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:17:08.538115 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e23c04-6334-4197-b2ad-ee96e7a21983 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:17:08.540670 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e23c04-6334-4197-b2ad-ee96e7a21983 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:17:08.541213 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1334/2649] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:17:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 15:17:08.549339 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf363c8-6f8c-49b5-96db-66efec2d8f49 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:17:08.549638 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf363c8-6f8c-49b5-96db-66efec2d8f49 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:17:08.550037 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf363c8-6f8c-49b5-96db-66efec2d8f49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:17:08.569429 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1316/2650] 10.222.0.46 () {60 vars in 1152 bytes} [Thu May 26 15:17:08 2022] POST /identity/v3/projects => generated 381 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 15:17:08.571494 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-2cf363c8-6f8c-49b5-96db-66efec2d8f49 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:17:08.572725 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-2cf363c8-6f8c-49b5-96db-66efec2d8f49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:17:08.573497 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e23c04-6334-4197-b2ad-ee96e7a21983 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:17:08.575655 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-c9e23c04-6334-4197-b2ad-ee96e7a21983 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:17:08.661643 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:17:08.662556 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e23c04-6334-4197-b2ad-ee96e7a21983 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:17:08.668313 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bece24d0-ecde-44d4-ac52-eeaa04552166 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:17:08.669024 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bece24d0-ecde-44d4-ac52-eeaa04552166 admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:17:08.669370 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bece24d0-ecde-44d4-ac52-eeaa04552166 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:17:08.669666 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bece24d0-ecde-44d4-ac52-eeaa04552166 admin admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:17:08.670494 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bece24d0-ecde-44d4-ac52-eeaa04552166 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:17:08.672531 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bece24d0-ecde-44d4-ac52-eeaa04552166 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:17:08.683907 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1335/2651] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:17:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:17:08.688943 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf363c8-6f8c-49b5-96db-66efec2d8f49 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:17:08.697522 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1317/2652] 10.222.0.46 () {60 vars in 1143 bytes} [Thu May 26 15:17:08 2022] POST /identity/v3/users => generated 297 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 15:17:08.698419 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf363c8-6f8c-49b5-96db-66efec2d8f49 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:17:08.699767 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bece24d0-ecde-44d4-ac52-eeaa04552166 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:17:08.704802 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d1a759-7291-4d8a-8820-bb655b59a3db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:17:08.705505 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d1a759-7291-4d8a-8820-bb655b59a3db admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:17:08.705827 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d1a759-7291-4d8a-8820-bb655b59a3db admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:17:08.706107 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d1a759-7291-4d8a-8820-bb655b59a3db admin admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:17:08.706867 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d1a759-7291-4d8a-8820-bb655b59a3db admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 15:17:08.707174 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d1a759-7291-4d8a-8820-bb655b59a3db admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:17:08.708770 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d1a759-7291-4d8a-8820-bb655b59a3db admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:17:08.710001 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bece24d0-ecde-44d4-ac52-eeaa04552166 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:17:08.715336 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1a93a0-7157-4688-87f0-6bb490ff783a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:17:08.715998 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1a93a0-7157-4688-87f0-6bb490ff783a admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:17:08.716289 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1a93a0-7157-4688-87f0-6bb490ff783a admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:17:08.716581 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1a93a0-7157-4688-87f0-6bb490ff783a admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:17:08.717317 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1a93a0-7157-4688-87f0-6bb490ff783a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:17:08.718964 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1a93a0-7157-4688-87f0-6bb490ff783a admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:17:08.724675 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1336/2653] 10.222.0.46 () {58 vars in 1152 bytes} [Thu May 26 15:17:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 15:17:08.725658 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d1a759-7291-4d8a-8820-bb655b59a3db admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:17:08.731051 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1318/2654] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 15:17:08 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 15:17:08.734850 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1a93a0-7157-4688-87f0-6bb490ff783a admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:17:08.736334 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d1a759-7291-4d8a-8820-bb655b59a3db admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:17:08.741251 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52424b21-a18b-4a1a-8e0e-3c9ddd9f28e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:17:08.742350 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52424b21-a18b-4a1a-8e0e-3c9ddd9f28e2 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:17:08.742350 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52424b21-a18b-4a1a-8e0e-3c9ddd9f28e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:17:08.742634 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52424b21-a18b-4a1a-8e0e-3c9ddd9f28e2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:17:08.743338 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52424b21-a18b-4a1a-8e0e-3c9ddd9f28e2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:17:08.745254 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52424b21-a18b-4a1a-8e0e-3c9ddd9f28e2 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:17:08.746259 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1a93a0-7157-4688-87f0-6bb490ff783a admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:17:08.752215 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b03413-e5d0-4ebf-8ff9-fd7863da130f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:17:08.753114 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b03413-e5d0-4ebf-8ff9-fd7863da130f admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:17:08.753405 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b03413-e5d0-4ebf-8ff9-fd7863da130f admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:17:08.753717 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b03413-e5d0-4ebf-8ff9-fd7863da130f admin admin] PATH_INFO: `/v3/projects/4bac0a6a7dd04a6eb1de2c65188d98cc/users/64e26080e65449aca46fcb1afdead4f2/roles/a2e551773cbd4df084020676865b8222` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:17:08.778741 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b03413-e5d0-4ebf-8ff9-fd7863da130f admin admin] RBAC: Authorizing `identity:create_grant(project_id=4bac0a6a7dd04a6eb1de2c65188d98cc, user_id=64e26080e65449aca46fcb1afdead4f2, role_id=a2e551773cbd4df084020676865b8222)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:17:08.781951 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b03413-e5d0-4ebf-8ff9-fd7863da130f admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:17:08.782394 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1337/2655] 10.222.0.46 () {60 vars in 1152 bytes} [Thu May 26 15:17:08 2022] POST /identity/v3/projects => generated 355 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 15:17:08.786554 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52424b21-a18b-4a1a-8e0e-3c9ddd9f28e2 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:17:08.797652 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1319/2656] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 15:17:08 2022] PUT /identity/v3/projects/4bac0a6a7dd04a6eb1de2c65188d98cc/users/64e26080e65449aca46fcb1afdead4f2/roles/a2e551773cbd4df084020676865b8222 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 15:17:08.801511 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b03413-e5d0-4ebf-8ff9-fd7863da130f admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:17:08.815147 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:17:08.856016 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52424b21-a18b-4a1a-8e0e-3c9ddd9f28e2 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:17:08.873105 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b03413-e5d0-4ebf-8ff9-fd7863da130f admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:17:08.877824 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba704af-8cb4-4a02-9581-64bedc43bf6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:17:08.878488 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba704af-8cb4-4a02-9581-64bedc43bf6b admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:17:08.878799 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba704af-8cb4-4a02-9581-64bedc43bf6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:17:08.879101 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba704af-8cb4-4a02-9581-64bedc43bf6b admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:17:08.880079 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba704af-8cb4-4a02-9581-64bedc43bf6b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:17:08.881443 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba704af-8cb4-4a02-9581-64bedc43bf6b admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:17:08.892993 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1320/2657] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 15:17:08 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 15:17:08.897596 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba704af-8cb4-4a02-9581-64bedc43bf6b admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:17:08.907812 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba704af-8cb4-4a02-9581-64bedc43bf6b admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:17:08.912592 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4735aef8-b4a9-46be-a30f-fe79595d8542 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:17:08.913490 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4735aef8-b4a9-46be-a30f-fe79595d8542 admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:17:08.914790 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4735aef8-b4a9-46be-a30f-fe79595d8542 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:17:08.915235 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4735aef8-b4a9-46be-a30f-fe79595d8542 admin admin] PATH_INFO: `/v3/projects/4bac0a6a7dd04a6eb1de2c65188d98cc/users/64e26080e65449aca46fcb1afdead4f2/roles/a2e551773cbd4df084020676865b8222` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:17:08.915407 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52fca5a-26bc-4c4c-a5ec-0d5ceb91848f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:17:08.916085 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52fca5a-26bc-4c4c-a5ec-0d5ceb91848f admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:17:08.916368 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52fca5a-26bc-4c4c-a5ec-0d5ceb91848f admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:17:08.916644 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52fca5a-26bc-4c4c-a5ec-0d5ceb91848f admin admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:17:08.917647 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4735aef8-b4a9-46be-a30f-fe79595d8542 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4bac0a6a7dd04a6eb1de2c65188d98cc, user_id=64e26080e65449aca46fcb1afdead4f2, role_id=a2e551773cbd4df084020676865b8222)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:17:08.918143 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52fca5a-26bc-4c4c-a5ec-0d5ceb91848f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:17:08.920443 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52fca5a-26bc-4c4c-a5ec-0d5ceb91848f admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:17:08.921604 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4735aef8-b4a9-46be-a30f-fe79595d8542 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:17:08.933013 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1321/2658] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 15:17:08 2022] PUT /identity/v3/projects/4bac0a6a7dd04a6eb1de2c65188d98cc/users/64e26080e65449aca46fcb1afdead4f2/roles/a2e551773cbd4df084020676865b8222 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 15:17:08.943680 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ab1311-9256-4728-836a-c8bb3eb568c1 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:17:08.943969 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ab1311-9256-4728-836a-c8bb3eb568c1 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:17:08.944243 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ab1311-9256-4728-836a-c8bb3eb568c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:17:08.950600 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1338/2659] 10.222.0.46 () {60 vars in 1143 bytes} [Thu May 26 15:17:08 2022] POST /identity/v3/users => generated 278 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 15:17:08.951613 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52fca5a-26bc-4c4c-a5ec-0d5ceb91848f admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:17:08.962125 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52fca5a-26bc-4c4c-a5ec-0d5ceb91848f admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:17:08.973949 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-f9ab1311-9256-4728-836a-c8bb3eb568c1 None None] MFA Rules not processed for user `64e26080e65449aca46fcb1afdead4f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:17:08.975169 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-f9ab1311-9256-4728-836a-c8bb3eb568c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:17:09.008427 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:17:09.021894 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6bc821-1ab9-4af0-bcff-5652a53e56ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:17:09.022567 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6bc821-1ab9-4af0-bcff-5652a53e56ef admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:17:09.022855 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6bc821-1ab9-4af0-bcff-5652a53e56ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:17:09.023795 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6bc821-1ab9-4af0-bcff-5652a53e56ef admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:17:09.024536 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6bc821-1ab9-4af0-bcff-5652a53e56ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:17:09.026171 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec6bc821-1ab9-4af0-bcff-5652a53e56ef admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:17:09.036640 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1339/2660] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 15:17:08 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 15:17:09.040399 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6bc821-1ab9-4af0-bcff-5652a53e56ef admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:17:09.046459 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1322/2661] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:17:08 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 15:17:09.050481 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec6bc821-1ab9-4af0-bcff-5652a53e56ef admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:17:09.055200 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bced93d0-24fb-4ebe-9a70-3548a799e653 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:17:09.055854 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bced93d0-24fb-4ebe-9a70-3548a799e653 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:17:09.056369 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bced93d0-24fb-4ebe-9a70-3548a799e653 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:17:09.056654 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bced93d0-24fb-4ebe-9a70-3548a799e653 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:17:09.057171 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ab1311-9256-4728-836a-c8bb3eb568c1 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:17:09.057550 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bced93d0-24fb-4ebe-9a70-3548a799e653 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:17:09.058587 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-f9ab1311-9256-4728-836a-c8bb3eb568c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:17:09.059335 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bced93d0-24fb-4ebe-9a70-3548a799e653 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:17:09.073112 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1340/2662] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 15:17:09 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 15:17:09.074305 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bced93d0-24fb-4ebe-9a70-3548a799e653 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:17:09.085305 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bced93d0-24fb-4ebe-9a70-3548a799e653 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:17:09.091730 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fdf5ad-fb99-4e98-9d7f-d04b6320c17e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:17:09.092603 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fdf5ad-fb99-4e98-9d7f-d04b6320c17e admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:17:09.092890 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fdf5ad-fb99-4e98-9d7f-d04b6320c17e admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:17:09.093174 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fdf5ad-fb99-4e98-9d7f-d04b6320c17e admin admin] PATH_INFO: `/v3/projects/d3c3b135e6cd411b9820f2a9dfb682a2/users/bf4bbdc8cb78401580d267b016c85597/roles/d4088d1845a24fbd9bd8975ce0b1d3d0` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:17:09.108675 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fdf5ad-fb99-4e98-9d7f-d04b6320c17e admin admin] RBAC: Authorizing `identity:create_grant(project_id=d3c3b135e6cd411b9820f2a9dfb682a2, user_id=bf4bbdc8cb78401580d267b016c85597, role_id=d4088d1845a24fbd9bd8975ce0b1d3d0)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:17:09.111360 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fdf5ad-fb99-4e98-9d7f-d04b6320c17e admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:17:09.120317 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ab1311-9256-4728-836a-c8bb3eb568c1 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:17:09.133814 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1341/2663] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 15:17:09 2022] PUT /identity/v3/projects/d3c3b135e6cd411b9820f2a9dfb682a2/users/bf4bbdc8cb78401580d267b016c85597/roles/d4088d1845a24fbd9bd8975ce0b1d3d0 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 15:17:09.135414 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2ad3b0-366f-4137-b48d-65d1bfa976d4 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:17:09.136390 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2ad3b0-366f-4137-b48d-65d1bfa976d4 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:17:09.136390 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2ad3b0-366f-4137-b48d-65d1bfa976d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:17:09.157212 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:17:09.164728 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-9d2ad3b0-366f-4137-b48d-65d1bfa976d4 None None] MFA Rules not processed for user `bf4bbdc8cb78401580d267b016c85597`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:17:09.166084 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-9d2ad3b0-366f-4137-b48d-65d1bfa976d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:17:09.175083 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b9bfe4-20ec-4f0e-9f4b-f9f1f790f8d0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5b5b35614a547d783e6e77ddb8c1c03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:17:09.175756 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b9bfe4-20ec-4f0e-9f4b-f9f1f790f8d0 service cinder] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:17:09.176038 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b9bfe4-20ec-4f0e-9f4b-f9f1f790f8d0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:17:09.176336 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b9bfe4-20ec-4f0e-9f4b-f9f1f790f8d0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:17:09.241721 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1342/2664] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:17:09 2022] POST /identity/v3/auth/tokens => generated 3222 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 15:17:09.249183 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2ad3b0-366f-4137-b48d-65d1bfa976d4 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:17:09.252346 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b9bfe4-20ec-4f0e-9f4b-f9f1f790f8d0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:17:09.254197 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b9bfe4-20ec-4f0e-9f4b-f9f1f790f8d0 service cinder] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:17:09.267869 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1323/2665] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:17:09 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 15:17:09.326708 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2ad3b0-366f-4137-b48d-65d1bfa976d4 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:17:09.331658 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71dd446-1fee-48f6-b0b4-77bce4e9337b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:17:09.332517 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71dd446-1fee-48f6-b0b4-77bce4e9337b admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:17:09.332874 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71dd446-1fee-48f6-b0b4-77bce4e9337b admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:17:09.333375 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71dd446-1fee-48f6-b0b4-77bce4e9337b admin admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:17:09.334207 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71dd446-1fee-48f6-b0b4-77bce4e9337b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:17:09.336500 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71dd446-1fee-48f6-b0b4-77bce4e9337b admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:17:09.370253 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1343/2666] 10.222.0.46 () {60 vars in 1152 bytes} [Thu May 26 15:17:09 2022] POST /identity/v3/projects => generated 357 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 15:17:09.374881 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b9bfe4-20ec-4f0e-9f4b-f9f1f790f8d0 service cinder] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:17:09.390675 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b9bfe4-20ec-4f0e-9f4b-f9f1f790f8d0 service cinder] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:17:09.410369 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71dd446-1fee-48f6-b0b4-77bce4e9337b admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:17:09.412294 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-a71dd446-1fee-48f6-b0b4-77bce4e9337b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:17:09.449306 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:17:09.476465 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f09fd97-2fd5-4bb3-9947-b453f1c6c63e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:17:09.477263 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f09fd97-2fd5-4bb3-9947-b453f1c6c63e admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:17:09.477988 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f09fd97-2fd5-4bb3-9947-b453f1c6c63e admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:17:09.478548 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f09fd97-2fd5-4bb3-9947-b453f1c6c63e admin admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:17:09.479642 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f09fd97-2fd5-4bb3-9947-b453f1c6c63e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:17:09.482187 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f09fd97-2fd5-4bb3-9947-b453f1c6c63e admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:17:09.499662 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71dd446-1fee-48f6-b0b4-77bce4e9337b admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:17:09.504823 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e023ce-992c-4fda-84a4-07a65dfed16d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:17:09.505757 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e023ce-992c-4fda-84a4-07a65dfed16d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:17:09.506043 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e023ce-992c-4fda-84a4-07a65dfed16d admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:17:09.506308 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e023ce-992c-4fda-84a4-07a65dfed16d admin admin] PATH_INFO: `/v3/users/cd74387e84d042c0a0bc5b151df3f6f6` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:17:09.508291 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e023ce-992c-4fda-84a4-07a65dfed16d admin admin] RBAC: Authorizing `identity:delete_user(user_id=cd74387e84d042c0a0bc5b151df3f6f6)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:17:09.509876 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1324/2667] 10.222.0.46 () {60 vars in 1143 bytes} [Thu May 26 15:17:09 2022] POST /identity/v3/users => generated 285 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 15:17:09.510617 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e023ce-992c-4fda-84a4-07a65dfed16d admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:17:09.514131 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f09fd97-2fd5-4bb3-9947-b453f1c6c63e admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:17:09.528754 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f09fd97-2fd5-4bb3-9947-b453f1c6c63e admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:17:09.539530 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445b288d-cb1d-47b3-ad71-0b33e76821ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:17:09.540337 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445b288d-cb1d-47b3-ad71-0b33e76821ef admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:17:09.540750 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445b288d-cb1d-47b3-ad71-0b33e76821ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:17:09.541308 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445b288d-cb1d-47b3-ad71-0b33e76821ef admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:17:09.542894 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445b288d-cb1d-47b3-ad71-0b33e76821ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:17:09.545211 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445b288d-cb1d-47b3-ad71-0b33e76821ef admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:17:09.559720 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1325/2668] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 15:17:09 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 15:17:09.564136 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445b288d-cb1d-47b3-ad71-0b33e76821ef admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:17:09.589235 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445b288d-cb1d-47b3-ad71-0b33e76821ef admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:17:09.594010 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e401acf5-ad54-4eda-84e2-1aad9d7b3157 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:17:09.595019 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e401acf5-ad54-4eda-84e2-1aad9d7b3157 admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:17:09.595303 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e401acf5-ad54-4eda-84e2-1aad9d7b3157 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:17:09.595605 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e401acf5-ad54-4eda-84e2-1aad9d7b3157 admin admin] PATH_INFO: `/v3/projects/5f990fe1a25740f4a655b0fa63c73ad4/users/062a6710d24f4d9cb3de4619777346bc/roles/a2e551773cbd4df084020676865b8222` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:17:09.595727 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-e2e023ce-992c-4fda-84a4-07a65dfed16d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cd74387e84d042c0a0bc5b151df3f6f6'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:17:09.607414 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-e2e023ce-992c-4fda-84a4-07a65dfed16d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cd74387e84d042c0a0bc5b151df3f6f6'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:17:09.608369 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-e2e023ce-992c-4fda-84a4-07a65dfed16d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cd74387e84d042c0a0bc5b151df3f6f6'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:17:09.627718 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e401acf5-ad54-4eda-84e2-1aad9d7b3157 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f990fe1a25740f4a655b0fa63c73ad4, user_id=062a6710d24f4d9cb3de4619777346bc, role_id=a2e551773cbd4df084020676865b8222)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:17:09.634659 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e401acf5-ad54-4eda-84e2-1aad9d7b3157 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:17:09.643195 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-e2e023ce-992c-4fda-84a4-07a65dfed16d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cd74387e84d042c0a0bc5b151df3f6f6'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:17:09.652670 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-e2e023ce-992c-4fda-84a4-07a65dfed16d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cd74387e84d042c0a0bc5b151df3f6f6'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:17:09.658234 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1326/2669] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 15:17:09 2022] PUT /identity/v3/projects/5f990fe1a25740f4a655b0fa63c73ad4/users/062a6710d24f4d9cb3de4619777346bc/roles/a2e551773cbd4df084020676865b8222 => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 15:17:09.659966 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1344/2670] 10.222.0.46 () {58 vars in 1223 bytes} [Thu May 26 15:17:09 2022] DELETE /identity/v3/users/cd74387e84d042c0a0bc5b151df3f6f6 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:17:09.661771 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e401acf5-ad54-4eda-84e2-1aad9d7b3157 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:17:09.664771 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bceb5b2b-f3da-4fce-b589-052809108406 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:17:09.665131 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bceb5b2b-f3da-4fce-b589-052809108406 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:17:09.665493 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bceb5b2b-f3da-4fce-b589-052809108406 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:17:09.665874 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-e401acf5-ad54-4eda-84e2-1aad9d7b3157 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:17:09.684831 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-bceb5b2b-f3da-4fce-b589-052809108406 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:17:09.686629 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-bceb5b2b-f3da-4fce-b589-052809108406 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:17:09.722064 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:17:09.752971 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e401acf5-ad54-4eda-84e2-1aad9d7b3157 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:17:09.776040 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1345/2671] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:17:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:17:09.776040 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bceb5b2b-f3da-4fce-b589-052809108406 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:17:09.776040 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-bceb5b2b-f3da-4fce-b589-052809108406 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:17:09.796517 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bceb5b2b-f3da-4fce-b589-052809108406 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:17:09.801362 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a653c4d-5f51-42ba-913b-979ace7a8f93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:17:09.802273 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a653c4d-5f51-42ba-913b-979ace7a8f93 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:17:09.802554 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a653c4d-5f51-42ba-913b-979ace7a8f93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:17:09.802824 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a653c4d-5f51-42ba-913b-979ace7a8f93 admin admin] PATH_INFO: `/v3/users/49188f7b627d4be799e805e48496bf9c` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:17:09.804014 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a653c4d-5f51-42ba-913b-979ace7a8f93 admin admin] RBAC: Authorizing `identity:delete_user(user_id=49188f7b627d4be799e805e48496bf9c)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:17:09.806611 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a653c4d-5f51-42ba-913b-979ace7a8f93 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:17:09.817943 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a82fc9d-efdd-4915-a079-04c187cfa78e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:17:09.818540 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a82fc9d-efdd-4915-a079-04c187cfa78e admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:17:09.818827 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a82fc9d-efdd-4915-a079-04c187cfa78e admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:17:09.819114 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a82fc9d-efdd-4915-a079-04c187cfa78e admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:17:09.819996 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a82fc9d-efdd-4915-a079-04c187cfa78e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:17:09.821889 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a82fc9d-efdd-4915-a079-04c187cfa78e admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:17:09.835188 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1327/2672] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 15:17:09 2022] GET /identity/v3/roles => generated 1420 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 15:17:09.836298 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a82fc9d-efdd-4915-a079-04c187cfa78e admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:17:09.838175 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-0a82fc9d-efdd-4915-a079-04c187cfa78e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:17:09.860382 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:17:09.870002 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-1a653c4d-5f51-42ba-913b-979ace7a8f93 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '49188f7b627d4be799e805e48496bf9c'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:17:09.878194 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-1a653c4d-5f51-42ba-913b-979ace7a8f93 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '49188f7b627d4be799e805e48496bf9c'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:17:09.878879 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-1a653c4d-5f51-42ba-913b-979ace7a8f93 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '49188f7b627d4be799e805e48496bf9c'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:17:09.894798 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-1a653c4d-5f51-42ba-913b-979ace7a8f93 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '49188f7b627d4be799e805e48496bf9c'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:17:09.902288 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a82fc9d-efdd-4915-a079-04c187cfa78e admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:17:09.902624 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-1a653c4d-5f51-42ba-913b-979ace7a8f93 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '49188f7b627d4be799e805e48496bf9c'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:17:09.906056 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40e45906-fef1-4125-81ce-0c95f520c23e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0156a2f6f3794f479e76b85bdbbe5c26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:17:09.906718 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e45906-fef1-4125-81ce-0c95f520c23e service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:17:09.907000 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e45906-fef1-4125-81ce-0c95f520c23e service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:17:09.907292 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40e45906-fef1-4125-81ce-0c95f520c23e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:17:09.909187 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-40e45906-fef1-4125-81ce-0c95f520c23e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:17:09.910996 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1346/2673] 10.222.0.46 () {58 vars in 1223 bytes} [Thu May 26 15:17:09 2022] DELETE /identity/v3/users/49188f7b627d4be799e805e48496bf9c => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:17:09.914087 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a653c4d-5f51-42ba-913b-979ace7a8f93 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:17:09.915515 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-1a653c4d-5f51-42ba-913b-979ace7a8f93 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:17:09.924566 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e45906-fef1-4125-81ce-0c95f520c23e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:17:09.927137 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40e45906-fef1-4125-81ce-0c95f520c23e service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:17:09.939961 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1328/2674] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:17:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 15:17:09.941428 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b6f419-262f-4d9f-808f-e3a4ca0be2ca None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:17:09.941761 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b6f419-262f-4d9f-808f-e3a4ca0be2ca None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:17:09.942040 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b6f419-262f-4d9f-808f-e3a4ca0be2ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:17:09.961652 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-05b6f419-262f-4d9f-808f-e3a4ca0be2ca None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:17:09.963021 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-05b6f419-262f-4d9f-808f-e3a4ca0be2ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:17:10.000925 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a653c4d-5f51-42ba-913b-979ace7a8f93 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:17:10.006495 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bfdfec-15d2-464b-98c2-7635b78aba90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:17:10.007605 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bfdfec-15d2-464b-98c2-7635b78aba90 admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:17:10.008035 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bfdfec-15d2-464b-98c2-7635b78aba90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:17:10.008455 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bfdfec-15d2-464b-98c2-7635b78aba90 admin admin] PATH_INFO: `/v3/projects/5f990fe1a25740f4a655b0fa63c73ad4/users/062a6710d24f4d9cb3de4619777346bc/roles/a2e551773cbd4df084020676865b8222` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:17:10.010988 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45bfdfec-15d2-464b-98c2-7635b78aba90 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f990fe1a25740f4a655b0fa63c73ad4, user_id=062a6710d24f4d9cb3de4619777346bc, role_id=a2e551773cbd4df084020676865b8222)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:17:10.014763 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45bfdfec-15d2-464b-98c2-7635b78aba90 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:17:10.027225 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1347/2675] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 15:17:09 2022] PUT /identity/v3/projects/5f990fe1a25740f4a655b0fa63c73ad4/users/062a6710d24f4d9cb3de4619777346bc/roles/a2e551773cbd4df084020676865b8222 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:17:10.038363 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37f2a2a-e81a-4037-891a-78cad9f5c050 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:17:10.039192 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37f2a2a-e81a-4037-891a-78cad9f5c050 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:17:10.039478 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37f2a2a-e81a-4037-891a-78cad9f5c050 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:17:10.056627 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1329/2676] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:17:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:17:10.067575 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b6f419-262f-4d9f-808f-e3a4ca0be2ca None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:17:10.070195 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-05b6f419-262f-4d9f-808f-e3a4ca0be2ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:17:10.071853 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-a37f2a2a-e81a-4037-891a-78cad9f5c050 None None] MFA Rules not processed for user `062a6710d24f4d9cb3de4619777346bc`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:17:10.073858 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-a37f2a2a-e81a-4037-891a-78cad9f5c050 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:17:10.116459 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:17:10.149107 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b6f419-262f-4d9f-808f-e3a4ca0be2ca None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:17:10.169093 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1348/2677] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:17:10 2022] POST /identity/v3/auth/tokens => generated 3291 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 15:17:10.180753 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37f2a2a-e81a-4037-891a-78cad9f5c050 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:17:10.182229 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-a37f2a2a-e81a-4037-891a-78cad9f5c050 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:17:10.212875 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2696f35b-83bb-4dab-98b9-231e4ac8e53c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0156a2f6f3794f479e76b85bdbbe5c26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:17:10.213939 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2696f35b-83bb-4dab-98b9-231e4ac8e53c service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:17:10.214433 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2696f35b-83bb-4dab-98b9-231e4ac8e53c service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:17:10.215479 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2696f35b-83bb-4dab-98b9-231e4ac8e53c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:17:10.233646 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:DELETE=5,keystone:SELECT=93,keystone:INSERT=2 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:17:10.261955 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37f2a2a-e81a-4037-891a-78cad9f5c050 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:17:10.266456 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e14a2cd-d422-460e-8b81-25071c30d9db service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5b5b35614a547d783e6e77ddb8c1c03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:17:10.267554 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e14a2cd-d422-460e-8b81-25071c30d9db service cinder] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:17:10.267814 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e14a2cd-d422-460e-8b81-25071c30d9db service cinder] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:17:10.268102 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e14a2cd-d422-460e-8b81-25071c30d9db service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:17:10.282735 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e14a2cd-d422-460e-8b81-25071c30d9db service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:17:10.284465 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e14a2cd-d422-460e-8b81-25071c30d9db service cinder] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:17:10.297933 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1349/2678] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:17:10 2022] GET /identity/v3/auth/tokens => generated 3291 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 15:17:10.298993 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e14a2cd-d422-460e-8b81-25071c30d9db service cinder] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:17:10.300319 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-2e14a2cd-d422-460e-8b81-25071c30d9db service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:17:10.318231 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2696f35b-83bb-4dab-98b9-231e4ac8e53c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:17:10.319696 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2696f35b-83bb-4dab-98b9-231e4ac8e53c service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:17:10.331039 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1330/2679] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:17:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 264 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 15:17:10.379063 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e14a2cd-d422-460e-8b81-25071c30d9db service cinder] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:17:10.385637 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb688ec-8023-4b13-acbe-77f39033b0b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:17:10.386743 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb688ec-8023-4b13-acbe-77f39033b0b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:17:10.387156 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb688ec-8023-4b13-acbe-77f39033b0b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:17:10.387594 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb688ec-8023-4b13-acbe-77f39033b0b3 admin admin] PATH_INFO: `/v3/projects/d2169b075ad8405fbbf81d8fc39b0b53` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:17:10.389916 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb688ec-8023-4b13-acbe-77f39033b0b3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d2169b075ad8405fbbf81d8fc39b0b53)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:17:10.392865 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb688ec-8023-4b13-acbe-77f39033b0b3 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:17:10.405405 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2696f35b-83bb-4dab-98b9-231e4ac8e53c service neutron] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:17:10.417713 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2696f35b-83bb-4dab-98b9-231e4ac8e53c service neutron] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:17:10.421972 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0882a683-41e3-4be0-bfda-342c2b1f1a72 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5b5b35614a547d783e6e77ddb8c1c03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:17:10.422733 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0882a683-41e3-4be0-bfda-342c2b1f1a72 service cinder] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:17:10.423028 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0882a683-41e3-4be0-bfda-342c2b1f1a72 service cinder] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:17:10.423311 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0882a683-41e3-4be0-bfda-342c2b1f1a72 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:17:10.425426 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-0882a683-41e3-4be0-bfda-342c2b1f1a72 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:17:10.475234 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-1cb688ec-8023-4b13-acbe-77f39033b0b3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd2169b075ad8405fbbf81d8fc39b0b53', 'request_id': 'req-1cb688ec-8023-4b13-acbe-77f39033b0b3'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:17:10.485541 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-1cb688ec-8023-4b13-acbe-77f39033b0b3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd2169b075ad8405fbbf81d8fc39b0b53', 'request_id': 'req-1cb688ec-8023-4b13-acbe-77f39033b0b3'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:17:10.493873 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-1cb688ec-8023-4b13-acbe-77f39033b0b3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd2169b075ad8405fbbf81d8fc39b0b53', 'request_id': 'req-1cb688ec-8023-4b13-acbe-77f39033b0b3'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:17:10.497773 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0882a683-41e3-4be0-bfda-342c2b1f1a72 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:17:10.499469 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0882a683-41e3-4be0-bfda-342c2b1f1a72 service cinder] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:17:10.510741 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-1cb688ec-8023-4b13-acbe-77f39033b0b3 admin admin] The token cache is being invalidate because project d2169b075ad8405fbbf81d8fc39b0b53 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 15:17:10.511195 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-1cb688ec-8023-4b13-acbe-77f39033b0b3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:17:10.511923 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1331/2680] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:17:10 2022] GET /identity/v3/auth/tokens => generated 3222 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 15:17:10.512051 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-1cb688ec-8023-4b13-acbe-77f39033b0b3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:17:10.513874 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1350/2681] 10.222.0.46 () {58 vars in 1232 bytes} [Thu May 26 15:17:10 2022] DELETE /identity/v3/projects/d2169b075ad8405fbbf81d8fc39b0b53 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:17:10.518981 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b730975-85b1-4dd0-a9f2-29f72e02d887 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:17:10.519323 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b730975-85b1-4dd0-a9f2-29f72e02d887 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:17:10.519609 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b730975-85b1-4dd0-a9f2-29f72e02d887 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:17:10.538629 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-5b730975-85b1-4dd0-a9f2-29f72e02d887 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:17:10.540046 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-5b730975-85b1-4dd0-a9f2-29f72e02d887 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:17:10.578530 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:17:10.621701 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1332/2682] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:17:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:17:10.629871 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e39316e-2536-473a-9014-6138bcf5947a None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:17:10.630059 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e39316e-2536-473a-9014-6138bcf5947a None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:17:10.630347 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e39316e-2536-473a-9014-6138bcf5947a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:17:10.679494 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-4e39316e-2536-473a-9014-6138bcf5947a None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:17:10.681385 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-4e39316e-2536-473a-9014-6138bcf5947a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:17:10.696092 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1351/2683] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:17:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:17:10.700901 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b730975-85b1-4dd0-a9f2-29f72e02d887 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:17:10.714559 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b730975-85b1-4dd0-a9f2-29f72e02d887 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:17:10.720842 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5edaf03-2871-443b-bbda-ab2f0b1f3abf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:17:10.721555 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5edaf03-2871-443b-bbda-ab2f0b1f3abf admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:17:10.721841 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5edaf03-2871-443b-bbda-ab2f0b1f3abf admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:17:10.722125 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5edaf03-2871-443b-bbda-ab2f0b1f3abf admin admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:17:10.722994 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5edaf03-2871-443b-bbda-ab2f0b1f3abf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 15:17:10.723448 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5edaf03-2871-443b-bbda-ab2f0b1f3abf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:17:10.725161 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5edaf03-2871-443b-bbda-ab2f0b1f3abf admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:17:10.739943 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1333/2684] 10.222.0.46 () {58 vars in 1152 bytes} [Thu May 26 15:17:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 15:17:10.744866 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e39316e-2536-473a-9014-6138bcf5947a None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:17:10.754482 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e39316e-2536-473a-9014-6138bcf5947a None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:17:10.759882 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780bb876-0b07-4795-a4f2-229225647b38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:17:10.760899 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780bb876-0b07-4795-a4f2-229225647b38 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:17:10.761032 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780bb876-0b07-4795-a4f2-229225647b38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:17:10.761511 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780bb876-0b07-4795-a4f2-229225647b38 admin admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:17:10.762067 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780bb876-0b07-4795-a4f2-229225647b38 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:17:10.764218 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780bb876-0b07-4795-a4f2-229225647b38 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:17:10.795434 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1352/2685] 10.222.0.46 () {60 vars in 1152 bytes} [Thu May 26 15:17:10 2022] POST /identity/v3/projects => generated 369 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 15:17:10.796530 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5edaf03-2871-443b-bbda-ab2f0b1f3abf admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:17:10.807303 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5edaf03-2871-443b-bbda-ab2f0b1f3abf admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:17:10.891061 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a285c7-61f9-41b8-af2e-aa172257477f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:17:10.891763 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a285c7-61f9-41b8-af2e-aa172257477f admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:17:10.892051 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a285c7-61f9-41b8-af2e-aa172257477f admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:17:10.892611 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a285c7-61f9-41b8-af2e-aa172257477f admin admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:17:10.893034 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a285c7-61f9-41b8-af2e-aa172257477f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:17:10.895061 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a285c7-61f9-41b8-af2e-aa172257477f admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:17:10.922193 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1334/2686] 10.222.0.46 () {60 vars in 1143 bytes} [Thu May 26 15:17:10 2022] POST /identity/v3/users => generated 285 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 15:17:10.926725 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780bb876-0b07-4795-a4f2-229225647b38 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:17:10.939598 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780bb876-0b07-4795-a4f2-229225647b38 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:17:10.944721 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d99c171-5d8d-46b5-8915-3c239b8213a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:17:10.945495 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d99c171-5d8d-46b5-8915-3c239b8213a3 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:17:10.945803 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d99c171-5d8d-46b5-8915-3c239b8213a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:17:10.946079 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d99c171-5d8d-46b5-8915-3c239b8213a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:17:10.946840 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d99c171-5d8d-46b5-8915-3c239b8213a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:17:10.948452 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d99c171-5d8d-46b5-8915-3c239b8213a3 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:17:10.955393 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a285c7-61f9-41b8-af2e-aa172257477f admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:17:10.957305 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-15a285c7-61f9-41b8-af2e-aa172257477f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:17:10.964656 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1353/2687] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 15:17:10 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 15:17:10.968892 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d99c171-5d8d-46b5-8915-3c239b8213a3 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:17:10.976809 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a285c7-61f9-41b8-af2e-aa172257477f admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:17:10.985197 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86dfdc14-77b2-4ee6-aeea-5c8d9ac393c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:17:10.988537 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dfdc14-77b2-4ee6-aeea-5c8d9ac393c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:17:10.988777 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dfdc14-77b2-4ee6-aeea-5c8d9ac393c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:17:10.989063 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dfdc14-77b2-4ee6-aeea-5c8d9ac393c1 admin admin] PATH_INFO: `/v3/projects/30f1cd47846941e0ab5e19d976a44d6c` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:17:10.989908 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d99c171-5d8d-46b5-8915-3c239b8213a3 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:17:10.991303 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86dfdc14-77b2-4ee6-aeea-5c8d9ac393c1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=30f1cd47846941e0ab5e19d976a44d6c)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:17:10.993250 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86dfdc14-77b2-4ee6-aeea-5c8d9ac393c1 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:17:11.000104 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6db8443-d692-49bb-b267-54c47ac0c0f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:17:11.000794 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6db8443-d692-49bb-b267-54c47ac0c0f1 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:17:11.001073 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6db8443-d692-49bb-b267-54c47ac0c0f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:17:11.001349 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6db8443-d692-49bb-b267-54c47ac0c0f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:17:11.002626 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6db8443-d692-49bb-b267-54c47ac0c0f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:17:11.003791 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6db8443-d692-49bb-b267-54c47ac0c0f1 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:17:11.014828 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1354/2688] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 15:17:10 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 15:17:11.018980 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6db8443-d692-49bb-b267-54c47ac0c0f1 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:17:11.030561 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6db8443-d692-49bb-b267-54c47ac0c0f1 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:17:11.035819 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f045012-49a1-4e2d-ad4d-6526aebf0280 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:17:11.036846 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f045012-49a1-4e2d-ad4d-6526aebf0280 admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:17:11.037150 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f045012-49a1-4e2d-ad4d-6526aebf0280 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:17:11.037404 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f045012-49a1-4e2d-ad4d-6526aebf0280 admin admin] PATH_INFO: `/v3/projects/78466c91318f41baa08bff10235adb75/users/993290e392434879a2a9a9c283b2caad/roles/d4088d1845a24fbd9bd8975ce0b1d3d0` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:17:11.053707 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f045012-49a1-4e2d-ad4d-6526aebf0280 admin admin] RBAC: Authorizing `identity:create_grant(project_id=78466c91318f41baa08bff10235adb75, user_id=993290e392434879a2a9a9c283b2caad, role_id=d4088d1845a24fbd9bd8975ce0b1d3d0)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:17:11.056461 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f045012-49a1-4e2d-ad4d-6526aebf0280 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:17:11.059371 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:17:11.060020 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-86dfdc14-77b2-4ee6-aeea-5c8d9ac393c1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '30f1cd47846941e0ab5e19d976a44d6c', 'request_id': 'req-86dfdc14-77b2-4ee6-aeea-5c8d9ac393c1'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:17:11.068533 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-86dfdc14-77b2-4ee6-aeea-5c8d9ac393c1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '30f1cd47846941e0ab5e19d976a44d6c', 'request_id': 'req-86dfdc14-77b2-4ee6-aeea-5c8d9ac393c1'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:17:11.078887 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-86dfdc14-77b2-4ee6-aeea-5c8d9ac393c1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '30f1cd47846941e0ab5e19d976a44d6c', 'request_id': 'req-86dfdc14-77b2-4ee6-aeea-5c8d9ac393c1'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:17:11.083666 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1355/2689] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 15:17:11 2022] PUT /identity/v3/projects/78466c91318f41baa08bff10235adb75/users/993290e392434879a2a9a9c283b2caad/roles/d4088d1845a24fbd9bd8975ce0b1d3d0 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 15:17:11.096055 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6698a38-46f4-4373-af60-e1605697babe None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:17:11.096319 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6698a38-46f4-4373-af60-e1605697babe None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:17:11.096593 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6698a38-46f4-4373-af60-e1605697babe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:17:11.103822 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-86dfdc14-77b2-4ee6-aeea-5c8d9ac393c1 admin admin] The token cache is being invalidate because project 30f1cd47846941e0ab5e19d976a44d6c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 15:17:11.104414 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-86dfdc14-77b2-4ee6-aeea-5c8d9ac393c1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:17:11.105990 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-86dfdc14-77b2-4ee6-aeea-5c8d9ac393c1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:17:11.107901 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1335/2690] 10.222.0.46 () {58 vars in 1232 bytes} [Thu May 26 15:17:10 2022] DELETE /identity/v3/projects/30f1cd47846941e0ab5e19d976a44d6c => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:17:11.113493 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86dfdc14-77b2-4ee6-aeea-5c8d9ac393c1 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:17:11.115166 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-86dfdc14-77b2-4ee6-aeea-5c8d9ac393c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:17:11.135879 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-c6698a38-46f4-4373-af60-e1605697babe None None] MFA Rules not processed for user `993290e392434879a2a9a9c283b2caad`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:17:11.137272 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-c6698a38-46f4-4373-af60-e1605697babe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:17:11.165055 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:17:11.209007 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86dfdc14-77b2-4ee6-aeea-5c8d9ac393c1 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:17:11.215194 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1356/2691] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:17:11 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 26 15:17:11.223216 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6698a38-46f4-4373-af60-e1605697babe None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:17:11.224687 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-c6698a38-46f4-4373-af60-e1605697babe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:17:11.283690 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b029b49-1bca-49e4-87ab-c83e5395e55f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:17:11.284711 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b029b49-1bca-49e4-87ab-c83e5395e55f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:17:11.284988 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b029b49-1bca-49e4-87ab-c83e5395e55f admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:17:11.285264 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b029b49-1bca-49e4-87ab-c83e5395e55f admin admin] PATH_INFO: `/v3/users/64e26080e65449aca46fcb1afdead4f2` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:17:11.286608 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b029b49-1bca-49e4-87ab-c83e5395e55f admin admin] RBAC: Authorizing `identity:delete_user(user_id=64e26080e65449aca46fcb1afdead4f2)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:17:11.288844 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b029b49-1bca-49e4-87ab-c83e5395e55f admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:17:11.307944 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6698a38-46f4-4373-af60-e1605697babe None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:17:11.312372 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6033fe39-15c2-4302-b14d-71bbb869b9f4 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0a436dbdee624f44ad76cca8416b0ad1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:17:11.313043 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6033fe39-15c2-4302-b14d-71bbb869b9f4 service glance] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:17:11.313626 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6033fe39-15c2-4302-b14d-71bbb869b9f4 service glance] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:17:11.313772 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6033fe39-15c2-4302-b14d-71bbb869b9f4 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:17:11.323903 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6033fe39-15c2-4302-b14d-71bbb869b9f4 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:17:11.325541 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6033fe39-15c2-4302-b14d-71bbb869b9f4 service glance] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:17:11.337513 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1357/2692] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:17:11 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 15:17:11.356321 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-1b029b49-1bca-49e4-87ab-c83e5395e55f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '64e26080e65449aca46fcb1afdead4f2'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:17:11.375056 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-1b029b49-1bca-49e4-87ab-c83e5395e55f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '64e26080e65449aca46fcb1afdead4f2'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:17:11.375864 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-1b029b49-1bca-49e4-87ab-c83e5395e55f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '64e26080e65449aca46fcb1afdead4f2'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:17:11.399002 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-1b029b49-1bca-49e4-87ab-c83e5395e55f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '64e26080e65449aca46fcb1afdead4f2'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:17:11.408653 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-1b029b49-1bca-49e4-87ab-c83e5395e55f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '64e26080e65449aca46fcb1afdead4f2'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:17:11.415640 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1336/2693] 10.222.0.46 () {58 vars in 1223 bytes} [Thu May 26 15:17:11 2022] DELETE /identity/v3/users/64e26080e65449aca46fcb1afdead4f2 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 15:17:11.791949 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6033fe39-15c2-4302-b14d-71bbb869b9f4 service glance] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:17:11.803645 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6033fe39-15c2-4302-b14d-71bbb869b9f4 service glance] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:17:11.890742 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23849014-f57c-4012-a449-2dfcdaaedbb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:17:11.891347 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23849014-f57c-4012-a449-2dfcdaaedbb5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:17:11.891645 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23849014-f57c-4012-a449-2dfcdaaedbb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:17:11.891922 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23849014-f57c-4012-a449-2dfcdaaedbb5 admin admin] PATH_INFO: `/v3/projects/4bac0a6a7dd04a6eb1de2c65188d98cc` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:17:11.895008 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23849014-f57c-4012-a449-2dfcdaaedbb5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4bac0a6a7dd04a6eb1de2c65188d98cc)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:17:11.897192 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23849014-f57c-4012-a449-2dfcdaaedbb5 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:17:11.950109 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:INSERT=1,keystone:SELECT=97,keystone:DELETE=2 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:17:11.986643 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-23849014-f57c-4012-a449-2dfcdaaedbb5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4bac0a6a7dd04a6eb1de2c65188d98cc', 'request_id': 'req-23849014-f57c-4012-a449-2dfcdaaedbb5'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:17:11.998897 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-23849014-f57c-4012-a449-2dfcdaaedbb5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4bac0a6a7dd04a6eb1de2c65188d98cc', 'request_id': 'req-23849014-f57c-4012-a449-2dfcdaaedbb5'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:17:12.009519 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-23849014-f57c-4012-a449-2dfcdaaedbb5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4bac0a6a7dd04a6eb1de2c65188d98cc', 'request_id': 'req-23849014-f57c-4012-a449-2dfcdaaedbb5'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:17:12.025203 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-23849014-f57c-4012-a449-2dfcdaaedbb5 admin admin] The token cache is being invalidate because project 4bac0a6a7dd04a6eb1de2c65188d98cc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 15:17:12.025903 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-23849014-f57c-4012-a449-2dfcdaaedbb5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:17:12.027440 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-23849014-f57c-4012-a449-2dfcdaaedbb5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:17:12.030134 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1358/2694] 10.222.0.46 () {58 vars in 1232 bytes} [Thu May 26 15:17:11 2022] DELETE /identity/v3/projects/4bac0a6a7dd04a6eb1de2c65188d98cc => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:17:12.035305 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5578c9d-0a31-40c8-a015-edfa26fe47b2 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:17:12.035942 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5578c9d-0a31-40c8-a015-edfa26fe47b2 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:17:12.035942 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5578c9d-0a31-40c8-a015-edfa26fe47b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:17:12.059646 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-c5578c9d-0a31-40c8-a015-edfa26fe47b2 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:17:12.065060 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-c5578c9d-0a31-40c8-a015-edfa26fe47b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:17:12.129318 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:17:12.140245 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1337/2695] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:17:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:17:12.150184 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47f00d7-6c1e-4094-93af-133d5edb5ef7 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:17:12.150470 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47f00d7-6c1e-4094-93af-133d5edb5ef7 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:17:12.150754 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47f00d7-6c1e-4094-93af-133d5edb5ef7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:17:12.169544 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-b47f00d7-6c1e-4094-93af-133d5edb5ef7 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:17:12.170799 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-b47f00d7-6c1e-4094-93af-133d5edb5ef7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:17:12.178840 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1359/2696] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:17:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:17:12.183504 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5578c9d-0a31-40c8-a015-edfa26fe47b2 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:17:12.193123 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5578c9d-0a31-40c8-a015-edfa26fe47b2 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:17:12.198282 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70003c9-f862-42f9-bacf-b1171128d365 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:17:12.198930 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70003c9-f862-42f9-bacf-b1171128d365 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:17:12.199212 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70003c9-f862-42f9-bacf-b1171128d365 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:17:12.199488 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70003c9-f862-42f9-bacf-b1171128d365 admin admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:17:12.200207 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70003c9-f862-42f9-bacf-b1171128d365 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 15:17:12.200513 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70003c9-f862-42f9-bacf-b1171128d365 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:17:12.202269 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70003c9-f862-42f9-bacf-b1171128d365 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:17:12.214136 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1338/2697] 10.222.0.46 () {58 vars in 1152 bytes} [Thu May 26 15:17:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 15:17:12.218843 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47f00d7-6c1e-4094-93af-133d5edb5ef7 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:17:12.228121 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47f00d7-6c1e-4094-93af-133d5edb5ef7 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:17:12.233264 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9e9ca7-8860-407a-8d3a-7eb93ae7da7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:17:12.233954 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9e9ca7-8860-407a-8d3a-7eb93ae7da7c admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:17:12.234243 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9e9ca7-8860-407a-8d3a-7eb93ae7da7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:17:12.234524 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9e9ca7-8860-407a-8d3a-7eb93ae7da7c admin admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:17:12.235220 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9e9ca7-8860-407a-8d3a-7eb93ae7da7c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:17:12.237143 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9e9ca7-8860-407a-8d3a-7eb93ae7da7c admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:17:12.273495 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1360/2698] 10.222.0.46 () {60 vars in 1152 bytes} [Thu May 26 15:17:12 2022] POST /identity/v3/projects => generated 383 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 15:17:12.276441 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70003c9-f862-42f9-bacf-b1171128d365 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:17:12.286524 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70003c9-f862-42f9-bacf-b1171128d365 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:17:12.342503 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4171fa84-1c33-4281-9eff-8fdad594824c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:17:12.343284 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4171fa84-1c33-4281-9eff-8fdad594824c admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:17:12.343564 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4171fa84-1c33-4281-9eff-8fdad594824c admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:17:12.343838 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4171fa84-1c33-4281-9eff-8fdad594824c admin admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:17:12.344533 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4171fa84-1c33-4281-9eff-8fdad594824c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:17:12.346528 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4171fa84-1c33-4281-9eff-8fdad594824c admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:17:12.377753 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1339/2699] 10.222.0.46 () {60 vars in 1143 bytes} [Thu May 26 15:17:12 2022] POST /identity/v3/users => generated 292 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 15:17:12.380028 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9e9ca7-8860-407a-8d3a-7eb93ae7da7c admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:17:12.393082 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9e9ca7-8860-407a-8d3a-7eb93ae7da7c admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:17:12.399282 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0863eafc-4141-4de9-8c2b-5996a17ec6d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:17:12.400041 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0863eafc-4141-4de9-8c2b-5996a17ec6d8 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:17:12.400447 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0863eafc-4141-4de9-8c2b-5996a17ec6d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:17:12.400825 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0863eafc-4141-4de9-8c2b-5996a17ec6d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:17:12.402332 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0863eafc-4141-4de9-8c2b-5996a17ec6d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:17:12.404144 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0863eafc-4141-4de9-8c2b-5996a17ec6d8 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:17:12.417132 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1361/2700] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 15:17:12 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 15:17:12.421337 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4171fa84-1c33-4281-9eff-8fdad594824c admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:17:12.432066 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4171fa84-1c33-4281-9eff-8fdad594824c admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:17:12.436667 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c733c92b-2d50-4ace-874f-a3fead556f78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:17:12.437326 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c733c92b-2d50-4ace-874f-a3fead556f78 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:17:12.437693 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c733c92b-2d50-4ace-874f-a3fead556f78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:17:12.437900 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c733c92b-2d50-4ace-874f-a3fead556f78 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:17:12.438625 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c733c92b-2d50-4ace-874f-a3fead556f78 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:17:12.440211 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c733c92b-2d50-4ace-874f-a3fead556f78 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:17:12.450949 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1340/2701] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 15:17:12 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 15:17:12.455088 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0863eafc-4141-4de9-8c2b-5996a17ec6d8 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:17:12.465363 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0863eafc-4141-4de9-8c2b-5996a17ec6d8 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:17:12.469925 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8c9e26-6852-4d47-a2f0-ab459ac36e4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:17:12.470874 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8c9e26-6852-4d47-a2f0-ab459ac36e4c admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:17:12.471172 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8c9e26-6852-4d47-a2f0-ab459ac36e4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:17:12.471445 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8c9e26-6852-4d47-a2f0-ab459ac36e4c admin admin] PATH_INFO: `/v3/projects/d0a5e84f3db74408b8b41156696b69f5/users/75066748ab7d4012a45b8e3bab3e3f48/roles/d4088d1845a24fbd9bd8975ce0b1d3d0` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:17:12.486871 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8c9e26-6852-4d47-a2f0-ab459ac36e4c admin admin] RBAC: Authorizing `identity:create_grant(project_id=d0a5e84f3db74408b8b41156696b69f5, user_id=75066748ab7d4012a45b8e3bab3e3f48, role_id=d4088d1845a24fbd9bd8975ce0b1d3d0)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:17:12.489580 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8c9e26-6852-4d47-a2f0-ab459ac36e4c admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:17:12.504636 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1362/2702] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 15:17:12 2022] PUT /identity/v3/projects/d0a5e84f3db74408b8b41156696b69f5/users/75066748ab7d4012a45b8e3bab3e3f48/roles/d4088d1845a24fbd9bd8975ce0b1d3d0 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:17:12.517369 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96e93d1-3770-470b-8a15-c9d85852ffca None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:17:12.517711 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96e93d1-3770-470b-8a15-c9d85852ffca None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:17:12.517990 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96e93d1-3770-470b-8a15-c9d85852ffca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:17:12.547712 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-a96e93d1-3770-470b-8a15-c9d85852ffca None None] MFA Rules not processed for user `75066748ab7d4012a45b8e3bab3e3f48`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:17:12.549152 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-a96e93d1-3770-470b-8a15-c9d85852ffca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:17:12.633100 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1341/2703] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:17:12 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:17:12.644396 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8c9e26-6852-4d47-a2f0-ab459ac36e4c admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:17:12.646235 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-cb8c9e26-6852-4d47-a2f0-ab459ac36e4c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:17:12.709006 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8c9e26-6852-4d47-a2f0-ab459ac36e4c admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:17:12.715140 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acac42c0-e870-4eb5-9c31-309301a86997 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5b5b35614a547d783e6e77ddb8c1c03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:17:12.715756 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acac42c0-e870-4eb5-9c31-309301a86997 service cinder] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:17:12.716069 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acac42c0-e870-4eb5-9c31-309301a86997 service cinder] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:17:12.716347 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acac42c0-e870-4eb5-9c31-309301a86997 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:17:12.725630 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acac42c0-e870-4eb5-9c31-309301a86997 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:17:12.727575 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acac42c0-e870-4eb5-9c31-309301a86997 service cinder] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:17:12.736692 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1363/2704] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:17:12 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 15:17:21.336927 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96e93d1-3770-470b-8a15-c9d85852ffca None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:17:21.433133 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:17:21.441063 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96e93d1-3770-470b-8a15-c9d85852ffca None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:17:21.464622 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6717c5c-3799-4cf8-820a-896a8eeba1b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:17:21.465919 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6717c5c-3799-4cf8-820a-896a8eeba1b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:17:21.465919 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6717c5c-3799-4cf8-820a-896a8eeba1b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:17:21.466408 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6717c5c-3799-4cf8-820a-896a8eeba1b9 admin admin] PATH_INFO: `/v3/users/75066748ab7d4012a45b8e3bab3e3f48` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:17:21.467661 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6717c5c-3799-4cf8-820a-896a8eeba1b9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=75066748ab7d4012a45b8e3bab3e3f48)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:17:21.470260 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6717c5c-3799-4cf8-820a-896a8eeba1b9 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:17:21.480771 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acac42c0-e870-4eb5-9c31-309301a86997 service cinder] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:17:21.482708 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-acac42c0-e870-4eb5-9c31-309301a86997 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:17:21.549154 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-e6717c5c-3799-4cf8-820a-896a8eeba1b9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '75066748ab7d4012a45b8e3bab3e3f48'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:17:21.559762 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-e6717c5c-3799-4cf8-820a-896a8eeba1b9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '75066748ab7d4012a45b8e3bab3e3f48'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:17:21.560971 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-e6717c5c-3799-4cf8-820a-896a8eeba1b9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '75066748ab7d4012a45b8e3bab3e3f48'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:17:21.587811 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acac42c0-e870-4eb5-9c31-309301a86997 service cinder] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:17:21.588414 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-e6717c5c-3799-4cf8-820a-896a8eeba1b9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '75066748ab7d4012a45b8e3bab3e3f48'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:17:21.592338 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adc1b66-73ba-4e24-999d-1588baa4100d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0156a2f6f3794f479e76b85bdbbe5c26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:17:21.593530 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adc1b66-73ba-4e24-999d-1588baa4100d service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:17:21.594004 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adc1b66-73ba-4e24-999d-1588baa4100d service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:17:21.594446 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adc1b66-73ba-4e24-999d-1588baa4100d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:17:21.597289 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-4adc1b66-73ba-4e24-999d-1588baa4100d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:17:21.601137 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-e6717c5c-3799-4cf8-820a-896a8eeba1b9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '75066748ab7d4012a45b8e3bab3e3f48'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:17:21.619689 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1342/2705] 10.222.0.46 () {58 vars in 1223 bytes} [Thu May 26 15:17:21 2022] DELETE /identity/v3/users/75066748ab7d4012a45b8e3bab3e3f48 => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:17:21.627560 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd604ec-52e8-431a-ae85-ca8c6a04203f None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:17:21.627845 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:17:21.628064 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd604ec-52e8-431a-ae85-ca8c6a04203f None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:17:21.628420 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd604ec-52e8-431a-ae85-ca8c6a04203f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:17:21.664200 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-ddd604ec-52e8-431a-ae85-ca8c6a04203f None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:17:21.666180 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-ddd604ec-52e8-431a-ae85-ca8c6a04203f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:17:21.748770 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4adc1b66-73ba-4e24-999d-1588baa4100d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:17:21.750319 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4adc1b66-73ba-4e24-999d-1588baa4100d service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:17:21.762108 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1364/2706] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:17:21 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 282 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 15:17:21.770525 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1343/2707] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:17:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:17:21.801347 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adc1b66-73ba-4e24-999d-1588baa4100d service neutron] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:17:21.802888 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-4adc1b66-73ba-4e24-999d-1588baa4100d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:17:21.818087 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adc1b66-73ba-4e24-999d-1588baa4100d service neutron] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:17:21.888076 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5acd82b-1bf0-4070-8df3-b78e89946b83 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0156a2f6f3794f479e76b85bdbbe5c26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:17:21.889132 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5acd82b-1bf0-4070-8df3-b78e89946b83 service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:17:21.889490 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5acd82b-1bf0-4070-8df3-b78e89946b83 service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:17:21.889822 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5acd82b-1bf0-4070-8df3-b78e89946b83 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:17:21.903983 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5acd82b-1bf0-4070-8df3-b78e89946b83 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:17:21.905708 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5acd82b-1bf0-4070-8df3-b78e89946b83 service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:17:21.919552 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1365/2708] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:17:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 15:17:22.695187 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd604ec-52e8-431a-ae85-ca8c6a04203f None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:17:22.697170 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-ddd604ec-52e8-431a-ae85-ca8c6a04203f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:17:22.726737 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd604ec-52e8-431a-ae85-ca8c6a04203f None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:17:22.742200 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adefb7f-be73-46bf-9df7-4dd93db91ce8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:17:22.743090 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adefb7f-be73-46bf-9df7-4dd93db91ce8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:17:22.743371 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adefb7f-be73-46bf-9df7-4dd93db91ce8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:17:22.743647 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adefb7f-be73-46bf-9df7-4dd93db91ce8 admin admin] PATH_INFO: `/v3/projects/d0a5e84f3db74408b8b41156696b69f5` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:17:22.745431 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6adefb7f-be73-46bf-9df7-4dd93db91ce8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d0a5e84f3db74408b8b41156696b69f5)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:17:22.747557 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6adefb7f-be73-46bf-9df7-4dd93db91ce8 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:17:22.835058 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-6adefb7f-be73-46bf-9df7-4dd93db91ce8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd0a5e84f3db74408b8b41156696b69f5', 'request_id': 'req-6adefb7f-be73-46bf-9df7-4dd93db91ce8'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:17:22.847417 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-6adefb7f-be73-46bf-9df7-4dd93db91ce8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd0a5e84f3db74408b8b41156696b69f5', 'request_id': 'req-6adefb7f-be73-46bf-9df7-4dd93db91ce8'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:17:22.856570 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-6adefb7f-be73-46bf-9df7-4dd93db91ce8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd0a5e84f3db74408b8b41156696b69f5', 'request_id': 'req-6adefb7f-be73-46bf-9df7-4dd93db91ce8'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:17:22.885386 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:17:22.901032 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5acd82b-1bf0-4070-8df3-b78e89946b83 service neutron] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:17:22.901895 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-6adefb7f-be73-46bf-9df7-4dd93db91ce8 admin admin] The token cache is being invalidate because project d0a5e84f3db74408b8b41156696b69f5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 15:17:22.901895 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-6adefb7f-be73-46bf-9df7-4dd93db91ce8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:17:22.902549 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-c5acd82b-1bf0-4070-8df3-b78e89946b83 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:17:22.910072 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-6adefb7f-be73-46bf-9df7-4dd93db91ce8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:17:22.912331 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1344/2709] 10.222.0.46 () {58 vars in 1232 bytes} [Thu May 26 15:17:22 2022] DELETE /identity/v3/projects/d0a5e84f3db74408b8b41156696b69f5 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:17:22.917409 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733bc899-ac38-431a-9b47-94b034a05835 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:17:22.917795 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733bc899-ac38-431a-9b47-94b034a05835 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:17:22.918236 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733bc899-ac38-431a-9b47-94b034a05835 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:17:22.919490 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5acd82b-1bf0-4070-8df3-b78e89946b83 service neutron] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:17:22.949227 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-733bc899-ac38-431a-9b47-94b034a05835 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:17:22.953082 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-733bc899-ac38-431a-9b47-94b034a05835 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:17:23.005740 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9e071a-00ca-401a-bb26-f8df7f1a2916 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0156a2f6f3794f479e76b85bdbbe5c26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:17:23.007571 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9e071a-00ca-401a-bb26-f8df7f1a2916 service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:17:23.008442 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9e071a-00ca-401a-bb26-f8df7f1a2916 service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:17:23.008726 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9e071a-00ca-401a-bb26-f8df7f1a2916 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:17:23.011464 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-ad9e071a-00ca-401a-bb26-f8df7f1a2916 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:17:23.095761 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1345/2710] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:17:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:17:23.097373 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f93228-70c8-447f-b6ab-44e1dd9c50d0 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:17:23.097688 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f93228-70c8-447f-b6ab-44e1dd9c50d0 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:17:23.098115 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f93228-70c8-447f-b6ab-44e1dd9c50d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:17:23.123607 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-b7f93228-70c8-447f-b6ab-44e1dd9c50d0 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:17:23.125007 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-b7f93228-70c8-447f-b6ab-44e1dd9c50d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:17:23.136645 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1346/2711] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:17:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:17:23.141080 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f93228-70c8-447f-b6ab-44e1dd9c50d0 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:17:23.153073 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f93228-70c8-447f-b6ab-44e1dd9c50d0 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:17:23.153396 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9e071a-00ca-401a-bb26-f8df7f1a2916 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:17:23.155122 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9e071a-00ca-401a-bb26-f8df7f1a2916 service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:17:23.158419 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c956263-9a19-4845-9fcd-8e1d0b5aa2c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:17:23.159501 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c956263-9a19-4845-9fcd-8e1d0b5aa2c3 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:17:23.159501 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c956263-9a19-4845-9fcd-8e1d0b5aa2c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:17:23.159897 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c956263-9a19-4845-9fcd-8e1d0b5aa2c3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:17:23.160720 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c956263-9a19-4845-9fcd-8e1d0b5aa2c3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 15:17:23.161104 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c956263-9a19-4845-9fcd-8e1d0b5aa2c3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:17:23.162827 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c956263-9a19-4845-9fcd-8e1d0b5aa2c3 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:17:23.186434 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1366/2712] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:17:22 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 286 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 15:17:23.190734 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1347/2713] 10.222.0.46 () {58 vars in 1152 bytes} [Thu May 26 15:17:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 15:17:23.192930 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9e071a-00ca-401a-bb26-f8df7f1a2916 service neutron] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:17:23.203547 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:17:23.215053 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9e071a-00ca-401a-bb26-f8df7f1a2916 service neutron] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:17:23.220581 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8970cae7-c42c-4b56-a697-71588ccf8017 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:17:23.221538 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8970cae7-c42c-4b56-a697-71588ccf8017 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:17:23.222003 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8970cae7-c42c-4b56-a697-71588ccf8017 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:17:23.222397 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8970cae7-c42c-4b56-a697-71588ccf8017 admin admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:17:23.223361 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8970cae7-c42c-4b56-a697-71588ccf8017 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:17:23.226208 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8970cae7-c42c-4b56-a697-71588ccf8017 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:17:23.317641 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1367/2714] 10.222.0.46 () {60 vars in 1152 bytes} [Thu May 26 15:17:23 2022] POST /identity/v3/projects => generated 385 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 15:17:23.320389 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c956263-9a19-4845-9fcd-8e1d0b5aa2c3 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:17:23.337158 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c956263-9a19-4845-9fcd-8e1d0b5aa2c3 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:17:23.456330 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27953fea-dc54-45f5-b568-c56e86a105dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:17:23.457073 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27953fea-dc54-45f5-b568-c56e86a105dc admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:17:23.457358 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27953fea-dc54-45f5-b568-c56e86a105dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:17:23.457749 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27953fea-dc54-45f5-b568-c56e86a105dc admin admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:17:23.458363 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27953fea-dc54-45f5-b568-c56e86a105dc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:17:23.463980 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27953fea-dc54-45f5-b568-c56e86a105dc admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:17:23.490361 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1348/2715] 10.222.0.46 () {60 vars in 1143 bytes} [Thu May 26 15:17:23 2022] POST /identity/v3/users => generated 293 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 15:17:23.495298 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8970cae7-c42c-4b56-a697-71588ccf8017 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:17:23.510588 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8970cae7-c42c-4b56-a697-71588ccf8017 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:17:23.517767 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2fba4f-c537-45cc-a3ad-098d221543dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:17:23.518439 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2fba4f-c537-45cc-a3ad-098d221543dc admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:17:23.518752 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2fba4f-c537-45cc-a3ad-098d221543dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:17:23.519029 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2fba4f-c537-45cc-a3ad-098d221543dc admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:17:23.519781 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2fba4f-c537-45cc-a3ad-098d221543dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:17:23.521437 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2fba4f-c537-45cc-a3ad-098d221543dc admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:17:23.536305 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1368/2716] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 15:17:23 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 15:17:23.540564 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27953fea-dc54-45f5-b568-c56e86a105dc admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:17:23.566345 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27953fea-dc54-45f5-b568-c56e86a105dc admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:17:23.583816 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f55ecf8-d18f-4876-b657-99b382c608b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:17:23.585242 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f55ecf8-d18f-4876-b657-99b382c608b7 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:17:23.585780 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f55ecf8-d18f-4876-b657-99b382c608b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:17:23.590136 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f55ecf8-d18f-4876-b657-99b382c608b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:17:23.591655 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f55ecf8-d18f-4876-b657-99b382c608b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:17:23.595435 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f55ecf8-d18f-4876-b657-99b382c608b7 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:17:23.625907 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1349/2717] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 15:17:23 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 15:17:23.632401 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2fba4f-c537-45cc-a3ad-098d221543dc admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:17:23.652480 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2fba4f-c537-45cc-a3ad-098d221543dc admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:17:23.673117 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349bdb1a-ca76-42aa-8b0e-a87fd47a3fe7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:17:23.674192 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349bdb1a-ca76-42aa-8b0e-a87fd47a3fe7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:17:23.674772 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349bdb1a-ca76-42aa-8b0e-a87fd47a3fe7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:17:23.675332 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349bdb1a-ca76-42aa-8b0e-a87fd47a3fe7 admin admin] PATH_INFO: `/v3/projects/f447e0c20b1c4c6b9ac9a943527ddaee/users/f71415d2d20d431f8ac602877d9dc7a1/roles/d4088d1845a24fbd9bd8975ce0b1d3d0` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:17:23.698436 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349bdb1a-ca76-42aa-8b0e-a87fd47a3fe7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f447e0c20b1c4c6b9ac9a943527ddaee, user_id=f71415d2d20d431f8ac602877d9dc7a1, role_id=d4088d1845a24fbd9bd8975ce0b1d3d0)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:17:23.701401 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349bdb1a-ca76-42aa-8b0e-a87fd47a3fe7 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:17:23.722826 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1369/2718] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 15:17:23 2022] PUT /identity/v3/projects/f447e0c20b1c4c6b9ac9a943527ddaee/users/f71415d2d20d431f8ac602877d9dc7a1/roles/d4088d1845a24fbd9bd8975ce0b1d3d0 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:17:23.798152 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900da728-61de-4b3c-a392-995bf8f7bd65 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:17:23.798785 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900da728-61de-4b3c-a392-995bf8f7bd65 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:17:23.798785 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900da728-61de-4b3c-a392-995bf8f7bd65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:17:23.833485 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-900da728-61de-4b3c-a392-995bf8f7bd65 None None] MFA Rules not processed for user `f71415d2d20d431f8ac602877d9dc7a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:17:23.834877 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-900da728-61de-4b3c-a392-995bf8f7bd65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:17:23.917304 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:17:23.961686 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1350/2719] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:17:23 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:17:23.968084 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349bdb1a-ca76-42aa-8b0e-a87fd47a3fe7 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:17:23.970252 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-349bdb1a-ca76-42aa-8b0e-a87fd47a3fe7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:17:24.081581 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349bdb1a-ca76-42aa-8b0e-a87fd47a3fe7 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:17:24.086013 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac553d9-67a7-4ec1-a05c-b46656f7318a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5b5b35614a547d783e6e77ddb8c1c03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:17:24.086694 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac553d9-67a7-4ec1-a05c-b46656f7318a service cinder] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:17:24.088007 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac553d9-67a7-4ec1-a05c-b46656f7318a service cinder] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:17:24.088007 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac553d9-67a7-4ec1-a05c-b46656f7318a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:17:24.101057 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac553d9-67a7-4ec1-a05c-b46656f7318a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:17:24.103847 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac553d9-67a7-4ec1-a05c-b46656f7318a service cinder] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:17:24.119834 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1370/2720] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:17:23 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 15:17:24.773344 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900da728-61de-4b3c-a392-995bf8f7bd65 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:17:24.775329 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-900da728-61de-4b3c-a392-995bf8f7bd65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:17:24.940969 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900da728-61de-4b3c-a392-995bf8f7bd65 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:17:24.962648 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf0a670-b0e4-451e-8e86-393db67e3ff9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6639353c70449d1b3d10cd6487c3b00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:17:24.963337 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf0a670-b0e4-451e-8e86-393db67e3ff9 service nova] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:17:24.964262 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf0a670-b0e4-451e-8e86-393db67e3ff9 service nova] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:17:24.964262 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf0a670-b0e4-451e-8e86-393db67e3ff9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:17:24.970518 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-3cf0a670-b0e4-451e-8e86-393db67e3ff9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:17:25.003786 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf0a670-b0e4-451e-8e86-393db67e3ff9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:17:25.005868 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf0a670-b0e4-451e-8e86-393db67e3ff9 service nova] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:17:25.038881 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1351/2721] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:17:24 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 257 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 15:17:27.092925 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac553d9-67a7-4ec1-a05c-b46656f7318a service cinder] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:17:27.185178 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac553d9-67a7-4ec1-a05c-b46656f7318a service cinder] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:17:27.191708 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07602eb-df9b-49e4-95cc-9bd4c3ca4c20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:17:27.192603 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07602eb-df9b-49e4-95cc-9bd4c3ca4c20 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:17:27.192893 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07602eb-df9b-49e4-95cc-9bd4c3ca4c20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:17:27.193176 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07602eb-df9b-49e4-95cc-9bd4c3ca4c20 admin admin] PATH_INFO: `/v3/users/f71415d2d20d431f8ac602877d9dc7a1` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:17:27.194940 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07602eb-df9b-49e4-95cc-9bd4c3ca4c20 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f71415d2d20d431f8ac602877d9dc7a1)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:17:27.196902 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07602eb-df9b-49e4-95cc-9bd4c3ca4c20 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:17:27.278123 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-d07602eb-df9b-49e4-95cc-9bd4c3ca4c20 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f71415d2d20d431f8ac602877d9dc7a1'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:17:27.281499 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:17:27.291818 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-d07602eb-df9b-49e4-95cc-9bd4c3ca4c20 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f71415d2d20d431f8ac602877d9dc7a1'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:17:27.294050 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-d07602eb-df9b-49e4-95cc-9bd4c3ca4c20 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f71415d2d20d431f8ac602877d9dc7a1'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:17:27.310839 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-d07602eb-df9b-49e4-95cc-9bd4c3ca4c20 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f71415d2d20d431f8ac602877d9dc7a1'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:17:27.324712 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-d07602eb-df9b-49e4-95cc-9bd4c3ca4c20 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f71415d2d20d431f8ac602877d9dc7a1'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:17:27.337137 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1371/2722] 10.222.0.46 () {58 vars in 1223 bytes} [Thu May 26 15:17:27 2022] DELETE /identity/v3/users/f71415d2d20d431f8ac602877d9dc7a1 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:17:27.339659 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b5fc5d-ae8b-444e-81d1-ae54f8a25b87 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:17:27.340153 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b5fc5d-ae8b-444e-81d1-ae54f8a25b87 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:17:27.340412 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b5fc5d-ae8b-444e-81d1-ae54f8a25b87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:17:27.364812 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-e7b5fc5d-ae8b-444e-81d1-ae54f8a25b87 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:17:27.367085 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-e7b5fc5d-ae8b-444e-81d1-ae54f8a25b87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:17:27.455857 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1352/2723] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:17:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:17:27.493488 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07602eb-df9b-49e4-95cc-9bd4c3ca4c20 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:17:27.495431 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-d07602eb-df9b-49e4-95cc-9bd4c3ca4c20 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:17:27.579308 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07602eb-df9b-49e4-95cc-9bd4c3ca4c20 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:17:27.584473 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ecdab6-42a8-486c-8b26-92bdbbd29370 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0156a2f6f3794f479e76b85bdbbe5c26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:17:27.585294 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ecdab6-42a8-486c-8b26-92bdbbd29370 service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:17:27.585730 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ecdab6-42a8-486c-8b26-92bdbbd29370 service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:17:27.586132 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ecdab6-42a8-486c-8b26-92bdbbd29370 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:17:27.628666 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ecdab6-42a8-486c-8b26-92bdbbd29370 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:17:27.630823 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ecdab6-42a8-486c-8b26-92bdbbd29370 service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:17:27.647824 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1372/2724] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:17:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 15:17:28.533272 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b5fc5d-ae8b-444e-81d1-ae54f8a25b87 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:17:28.535100 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-e7b5fc5d-ae8b-444e-81d1-ae54f8a25b87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:17:28.642725 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b5fc5d-ae8b-444e-81d1-ae54f8a25b87 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:17:28.648882 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102c026a-1c75-4c67-97a8-236b5f914893 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf55a00e3b084aa3b22c25a90e6a773c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:17:28.649758 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102c026a-1c75-4c67-97a8-236b5f914893 service placement] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:17:28.649953 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102c026a-1c75-4c67-97a8-236b5f914893 service placement] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:17:28.650235 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102c026a-1c75-4c67-97a8-236b5f914893 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:17:28.652743 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-102c026a-1c75-4c67-97a8-236b5f914893 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:17:28.676893 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102c026a-1c75-4c67-97a8-236b5f914893 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:17:28.683876 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102c026a-1c75-4c67-97a8-236b5f914893 service placement] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:17:28.697116 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1353/2725] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:17:28 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 15:17:28.808488 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ecdab6-42a8-486c-8b26-92bdbbd29370 service neutron] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:17:28.810150 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-68ecdab6-42a8-486c-8b26-92bdbbd29370 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:17:28.826426 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ecdab6-42a8-486c-8b26-92bdbbd29370 service neutron] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:17:28.831285 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491c29d1-3862-49df-be1f-babaff5e3d31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:17:28.832418 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491c29d1-3862-49df-be1f-babaff5e3d31 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:17:28.832866 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491c29d1-3862-49df-be1f-babaff5e3d31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:17:28.833582 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491c29d1-3862-49df-be1f-babaff5e3d31 admin admin] PATH_INFO: `/v3/projects/f447e0c20b1c4c6b9ac9a943527ddaee` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:17:28.835632 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491c29d1-3862-49df-be1f-babaff5e3d31 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f447e0c20b1c4c6b9ac9a943527ddaee)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:17:28.839420 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491c29d1-3862-49df-be1f-babaff5e3d31 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:17:28.934117 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-491c29d1-3862-49df-be1f-babaff5e3d31 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f447e0c20b1c4c6b9ac9a943527ddaee', 'request_id': 'req-491c29d1-3862-49df-be1f-babaff5e3d31'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:17:28.941720 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-491c29d1-3862-49df-be1f-babaff5e3d31 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f447e0c20b1c4c6b9ac9a943527ddaee', 'request_id': 'req-491c29d1-3862-49df-be1f-babaff5e3d31'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:17:28.952375 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-491c29d1-3862-49df-be1f-babaff5e3d31 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f447e0c20b1c4c6b9ac9a943527ddaee', 'request_id': 'req-491c29d1-3862-49df-be1f-babaff5e3d31'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:17:28.968550 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-491c29d1-3862-49df-be1f-babaff5e3d31 admin admin] The token cache is being invalidate because project f447e0c20b1c4c6b9ac9a943527ddaee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 15:17:28.968840 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-491c29d1-3862-49df-be1f-babaff5e3d31 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:17:28.969544 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-491c29d1-3862-49df-be1f-babaff5e3d31 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:17:28.971480 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1373/2726] 10.222.0.46 () {58 vars in 1232 bytes} [Thu May 26 15:17:28 2022] DELETE /identity/v3/projects/f447e0c20b1c4c6b9ac9a943527ddaee => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:17:28.977112 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6d2ccf-94e1-418d-b8f6-c8037fe82f9d None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:17:28.977404 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6d2ccf-94e1-418d-b8f6-c8037fe82f9d None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:17:28.977720 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6d2ccf-94e1-418d-b8f6-c8037fe82f9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:17:28.984380 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=100 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:17:29.001270 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-7f6d2ccf-94e1-418d-b8f6-c8037fe82f9d None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:17:29.002544 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-7f6d2ccf-94e1-418d-b8f6-c8037fe82f9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:17:29.089580 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1354/2727] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:17:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:17:29.098898 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fc4137-0197-4699-805c-6be3a642a17c None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:17:29.099217 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fc4137-0197-4699-805c-6be3a642a17c None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:17:29.099549 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fc4137-0197-4699-805c-6be3a642a17c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:17:29.127162 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-c0fc4137-0197-4699-805c-6be3a642a17c None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:17:29.128481 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-c0fc4137-0197-4699-805c-6be3a642a17c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:17:29.140261 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1374/2728] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:17:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:17:29.144711 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6d2ccf-94e1-418d-b8f6-c8037fe82f9d None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:17:29.156127 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6d2ccf-94e1-418d-b8f6-c8037fe82f9d None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:17:29.163810 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89eaf18b-8923-4765-b991-78d001709c3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:17:29.165292 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89eaf18b-8923-4765-b991-78d001709c3a admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:17:29.165618 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89eaf18b-8923-4765-b991-78d001709c3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:17:29.165901 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89eaf18b-8923-4765-b991-78d001709c3a admin admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:17:29.166651 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89eaf18b-8923-4765-b991-78d001709c3a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 15:17:29.166961 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89eaf18b-8923-4765-b991-78d001709c3a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:17:29.168629 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89eaf18b-8923-4765-b991-78d001709c3a admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:17:29.197146 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1355/2729] 10.222.0.46 () {58 vars in 1152 bytes} [Thu May 26 15:17:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 26 15:17:29.203941 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fc4137-0197-4699-805c-6be3a642a17c None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:17:29.257254 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fc4137-0197-4699-805c-6be3a642a17c None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:17:29.263242 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be035edd-02ad-4d22-9640-9f1a36c7841d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:17:29.264390 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be035edd-02ad-4d22-9640-9f1a36c7841d admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:17:29.264870 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be035edd-02ad-4d22-9640-9f1a36c7841d admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:17:29.265305 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be035edd-02ad-4d22-9640-9f1a36c7841d admin admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:17:29.266402 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be035edd-02ad-4d22-9640-9f1a36c7841d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:17:29.268667 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be035edd-02ad-4d22-9640-9f1a36c7841d admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:17:29.325106 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1375/2730] 10.222.0.46 () {60 vars in 1152 bytes} [Thu May 26 15:17:29 2022] POST /identity/v3/projects => generated 377 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 15:17:29.330575 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89eaf18b-8923-4765-b991-78d001709c3a admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:17:29.344488 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89eaf18b-8923-4765-b991-78d001709c3a admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:17:29.408898 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec647bf5-0efd-4b14-ad30-280e48006974 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:17:29.409530 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec647bf5-0efd-4b14-ad30-280e48006974 admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:17:29.409816 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec647bf5-0efd-4b14-ad30-280e48006974 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:17:29.410119 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec647bf5-0efd-4b14-ad30-280e48006974 admin admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:17:29.410813 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec647bf5-0efd-4b14-ad30-280e48006974 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:17:29.413008 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec647bf5-0efd-4b14-ad30-280e48006974 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:17:29.438475 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1356/2731] 10.222.0.46 () {60 vars in 1143 bytes} [Thu May 26 15:17:29 2022] POST /identity/v3/users => generated 289 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 15:17:29.442599 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be035edd-02ad-4d22-9640-9f1a36c7841d admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:17:29.454949 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be035edd-02ad-4d22-9640-9f1a36c7841d admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:17:29.459871 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f60afc4-50ae-453f-ac4a-5ac8f8672ecf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:17:29.460619 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f60afc4-50ae-453f-ac4a-5ac8f8672ecf admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:17:29.460904 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f60afc4-50ae-453f-ac4a-5ac8f8672ecf admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:17:29.461183 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f60afc4-50ae-453f-ac4a-5ac8f8672ecf admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:17:29.462122 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f60afc4-50ae-453f-ac4a-5ac8f8672ecf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:17:29.463804 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f60afc4-50ae-453f-ac4a-5ac8f8672ecf admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:17:29.475197 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1376/2732] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 15:17:29 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 15:17:29.479284 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec647bf5-0efd-4b14-ad30-280e48006974 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:17:29.489569 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec647bf5-0efd-4b14-ad30-280e48006974 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:17:29.495586 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75f44f1-cf59-4727-9868-49387a563b1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:17:29.496275 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75f44f1-cf59-4727-9868-49387a563b1b admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:17:29.496566 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75f44f1-cf59-4727-9868-49387a563b1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:17:29.496880 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75f44f1-cf59-4727-9868-49387a563b1b admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:17:29.497632 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75f44f1-cf59-4727-9868-49387a563b1b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:17:29.499338 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75f44f1-cf59-4727-9868-49387a563b1b admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:17:29.516238 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1357/2733] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 15:17:29 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 15:17:29.520812 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f60afc4-50ae-453f-ac4a-5ac8f8672ecf admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:17:29.531370 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f60afc4-50ae-453f-ac4a-5ac8f8672ecf admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:17:29.536063 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103f02b0-b781-4c3b-8af7-e76b63b5cd91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:17:29.536954 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103f02b0-b781-4c3b-8af7-e76b63b5cd91 admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:17:29.537239 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103f02b0-b781-4c3b-8af7-e76b63b5cd91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:17:29.537564 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103f02b0-b781-4c3b-8af7-e76b63b5cd91 admin admin] PATH_INFO: `/v3/projects/0b161426dbdc44ff8e52676e1bc21558/users/176fec434836494a9af12c96be3a8eb0/roles/d4088d1845a24fbd9bd8975ce0b1d3d0` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:17:29.549543 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:DELETE=10,keystone:SELECT=88,keystone:INSERT=2 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:17:29.554937 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103f02b0-b781-4c3b-8af7-e76b63b5cd91 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b161426dbdc44ff8e52676e1bc21558, user_id=176fec434836494a9af12c96be3a8eb0, role_id=d4088d1845a24fbd9bd8975ce0b1d3d0)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:17:29.557957 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103f02b0-b781-4c3b-8af7-e76b63b5cd91 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:17:29.576545 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1377/2734] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 15:17:29 2022] PUT /identity/v3/projects/0b161426dbdc44ff8e52676e1bc21558/users/176fec434836494a9af12c96be3a8eb0/roles/d4088d1845a24fbd9bd8975ce0b1d3d0 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:17:29.582175 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001248ca-afa4-4909-95dd-a57bd37dbc2c None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:17:29.582654 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001248ca-afa4-4909-95dd-a57bd37dbc2c None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:17:29.582742 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001248ca-afa4-4909-95dd-a57bd37dbc2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:17:29.603620 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-001248ca-afa4-4909-95dd-a57bd37dbc2c None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:17:29.605630 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-001248ca-afa4-4909-95dd-a57bd37dbc2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:17:29.676670 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1358/2735] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:17:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:17:29.686900 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103f02b0-b781-4c3b-8af7-e76b63b5cd91 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:17:29.688648 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-103f02b0-b781-4c3b-8af7-e76b63b5cd91 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:17:29.749008 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103f02b0-b781-4c3b-8af7-e76b63b5cd91 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:17:29.752742 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a813ed4-d15f-4e6e-a9c4-4ec80ed54691 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0156a2f6f3794f479e76b85bdbbe5c26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:17:29.753385 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a813ed4-d15f-4e6e-a9c4-4ec80ed54691 service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:17:29.753701 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a813ed4-d15f-4e6e-a9c4-4ec80ed54691 service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:17:29.753981 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a813ed4-d15f-4e6e-a9c4-4ec80ed54691 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:17:29.765378 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a813ed4-d15f-4e6e-a9c4-4ec80ed54691 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:17:29.766859 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a813ed4-d15f-4e6e-a9c4-4ec80ed54691 service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:17:29.778723 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1378/2736] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:17:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 15:17:30.274312 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001248ca-afa4-4909-95dd-a57bd37dbc2c None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:17:30.275765 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-001248ca-afa4-4909-95dd-a57bd37dbc2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:17:30.288036 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001248ca-afa4-4909-95dd-a57bd37dbc2c None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:17:30.293338 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9302d3-fcf8-4071-85f2-5e1fbbdc2ac0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:17:30.294344 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9302d3-fcf8-4071-85f2-5e1fbbdc2ac0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:17:30.294629 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9302d3-fcf8-4071-85f2-5e1fbbdc2ac0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:17:30.294909 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9302d3-fcf8-4071-85f2-5e1fbbdc2ac0 admin admin] PATH_INFO: `/v3/users/bf4bbdc8cb78401580d267b016c85597` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:17:30.297066 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9302d3-fcf8-4071-85f2-5e1fbbdc2ac0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bf4bbdc8cb78401580d267b016c85597)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:17:30.299114 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9302d3-fcf8-4071-85f2-5e1fbbdc2ac0 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:17:30.302443 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:17:30.393635 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-5e9302d3-fcf8-4071-85f2-5e1fbbdc2ac0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bf4bbdc8cb78401580d267b016c85597'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:17:30.404164 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-5e9302d3-fcf8-4071-85f2-5e1fbbdc2ac0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bf4bbdc8cb78401580d267b016c85597'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:17:30.406313 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-5e9302d3-fcf8-4071-85f2-5e1fbbdc2ac0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bf4bbdc8cb78401580d267b016c85597'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:17:30.425663 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-5e9302d3-fcf8-4071-85f2-5e1fbbdc2ac0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bf4bbdc8cb78401580d267b016c85597'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:17:30.436209 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-5e9302d3-fcf8-4071-85f2-5e1fbbdc2ac0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bf4bbdc8cb78401580d267b016c85597'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:17:30.448181 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1359/2737] 10.222.0.46 () {58 vars in 1223 bytes} [Thu May 26 15:17:30 2022] DELETE /identity/v3/users/bf4bbdc8cb78401580d267b016c85597 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 15:17:30.455537 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bff23d-51cc-4daf-82bd-3298863d4551 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:17:30.456011 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bff23d-51cc-4daf-82bd-3298863d4551 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:17:30.456358 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04bff23d-51cc-4daf-82bd-3298863d4551 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:17:30.488873 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-04bff23d-51cc-4daf-82bd-3298863d4551 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:17:30.493626 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-04bff23d-51cc-4daf-82bd-3298863d4551 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:17:30.607075 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1379/2738] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:17:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:17:30.633598 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9302d3-fcf8-4071-85f2-5e1fbbdc2ac0 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:17:30.635637 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-5e9302d3-fcf8-4071-85f2-5e1fbbdc2ac0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:17:30.741826 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9302d3-fcf8-4071-85f2-5e1fbbdc2ac0 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:17:30.759338 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bff23d-51cc-4daf-82bd-3298863d4551 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:17:30.759773 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34515c66-c622-4ff9-b962-60b6e47ea338 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0156a2f6f3794f479e76b85bdbbe5c26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:17:30.761000 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34515c66-c622-4ff9-b962-60b6e47ea338 service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:17:30.761121 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-04bff23d-51cc-4daf-82bd-3298863d4551 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:17:30.761898 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34515c66-c622-4ff9-b962-60b6e47ea338 service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:17:30.762717 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34515c66-c622-4ff9-b962-60b6e47ea338 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:17:30.776175 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04bff23d-51cc-4daf-82bd-3298863d4551 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:17:30.779007 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34515c66-c622-4ff9-b962-60b6e47ea338 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:17:30.780715 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536f2b4c-c66b-4ff7-a9fe-d3a89fa5b2fa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0156a2f6f3794f479e76b85bdbbe5c26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:17:30.781313 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34515c66-c622-4ff9-b962-60b6e47ea338 service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:17:30.781822 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536f2b4c-c66b-4ff7-a9fe-d3a89fa5b2fa service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:17:30.782296 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536f2b4c-c66b-4ff7-a9fe-d3a89fa5b2fa service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:17:30.782832 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536f2b4c-c66b-4ff7-a9fe-d3a89fa5b2fa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:17:30.784983 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-536f2b4c-c66b-4ff7-a9fe-d3a89fa5b2fa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:17:30.797712 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1360/2739] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:17:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 15:17:30.801949 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536f2b4c-c66b-4ff7-a9fe-d3a89fa5b2fa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:17:30.803964 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536f2b4c-c66b-4ff7-a9fe-d3a89fa5b2fa service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:17:30.818046 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1380/2740] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:17:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 15:17:31.680662 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34515c66-c622-4ff9-b962-60b6e47ea338 service neutron] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:17:31.682811 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-34515c66-c622-4ff9-b962-60b6e47ea338 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:17:31.715910 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34515c66-c622-4ff9-b962-60b6e47ea338 service neutron] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:17:31.724197 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1be768-0d28-40af-a7b0-edad931fa105 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:17:31.726350 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1be768-0d28-40af-a7b0-edad931fa105 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:17:31.727017 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1be768-0d28-40af-a7b0-edad931fa105 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:17:31.727700 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1be768-0d28-40af-a7b0-edad931fa105 admin admin] PATH_INFO: `/v3/projects/d3c3b135e6cd411b9820f2a9dfb682a2` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:17:31.729741 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1be768-0d28-40af-a7b0-edad931fa105 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d3c3b135e6cd411b9820f2a9dfb682a2)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:17:31.731884 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1be768-0d28-40af-a7b0-edad931fa105 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:17:31.819331 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-0d1be768-0d28-40af-a7b0-edad931fa105 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd3c3b135e6cd411b9820f2a9dfb682a2', 'request_id': 'req-0d1be768-0d28-40af-a7b0-edad931fa105'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:17:31.834327 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-0d1be768-0d28-40af-a7b0-edad931fa105 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd3c3b135e6cd411b9820f2a9dfb682a2', 'request_id': 'req-0d1be768-0d28-40af-a7b0-edad931fa105'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:17:31.857711 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-0d1be768-0d28-40af-a7b0-edad931fa105 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd3c3b135e6cd411b9820f2a9dfb682a2', 'request_id': 'req-0d1be768-0d28-40af-a7b0-edad931fa105'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:17:31.876540 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-0d1be768-0d28-40af-a7b0-edad931fa105 admin admin] The token cache is being invalidate because project d3c3b135e6cd411b9820f2a9dfb682a2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 15:17:31.877367 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-0d1be768-0d28-40af-a7b0-edad931fa105 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:17:31.878442 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-0d1be768-0d28-40af-a7b0-edad931fa105 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:17:31.884396 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1361/2741] 10.222.0.46 () {58 vars in 1232 bytes} [Thu May 26 15:17:31 2022] DELETE /identity/v3/projects/d3c3b135e6cd411b9820f2a9dfb682a2 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:17:31.884645 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536f2b4c-c66b-4ff7-a9fe-d3a89fa5b2fa service neutron] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:17:31.895720 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536f2b4c-c66b-4ff7-a9fe-d3a89fa5b2fa service neutron] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:17:31.982986 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5926dd20-5bb0-4ab1-9771-646f8cd50dfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:17:31.983960 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5926dd20-5bb0-4ab1-9771-646f8cd50dfa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:17:31.984312 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5926dd20-5bb0-4ab1-9771-646f8cd50dfa admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:17:31.984656 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5926dd20-5bb0-4ab1-9771-646f8cd50dfa admin admin] PATH_INFO: `/v3/users/062a6710d24f4d9cb3de4619777346bc` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:17:31.985966 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5926dd20-5bb0-4ab1-9771-646f8cd50dfa admin admin] RBAC: Authorizing `identity:delete_user(user_id=062a6710d24f4d9cb3de4619777346bc)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:17:31.988397 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5926dd20-5bb0-4ab1-9771-646f8cd50dfa admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:17:32.015806 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:17:32.065200 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-5926dd20-5bb0-4ab1-9771-646f8cd50dfa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '062a6710d24f4d9cb3de4619777346bc'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:17:32.075854 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-5926dd20-5bb0-4ab1-9771-646f8cd50dfa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '062a6710d24f4d9cb3de4619777346bc'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:17:32.076890 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-5926dd20-5bb0-4ab1-9771-646f8cd50dfa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '062a6710d24f4d9cb3de4619777346bc'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:17:32.101481 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-5926dd20-5bb0-4ab1-9771-646f8cd50dfa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '062a6710d24f4d9cb3de4619777346bc'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:17:32.113026 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-5926dd20-5bb0-4ab1-9771-646f8cd50dfa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '062a6710d24f4d9cb3de4619777346bc'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:17:32.123592 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1381/2742] 10.222.0.46 () {58 vars in 1223 bytes} [Thu May 26 15:17:31 2022] DELETE /identity/v3/users/062a6710d24f4d9cb3de4619777346bc => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:17:32.828344 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1be768-0d28-40af-a7b0-edad931fa105 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:17:32.830542 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-0d1be768-0d28-40af-a7b0-edad931fa105 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:17:32.868871 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1be768-0d28-40af-a7b0-edad931fa105 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:17:32.874955 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:17:32.971047 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5617f0f5-0e55-44ea-878e-2439620cdbc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:17:32.972047 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5617f0f5-0e55-44ea-878e-2439620cdbc6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:17:32.972333 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5617f0f5-0e55-44ea-878e-2439620cdbc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:17:32.972614 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5617f0f5-0e55-44ea-878e-2439620cdbc6 admin admin] PATH_INFO: `/v3/projects/5f990fe1a25740f4a655b0fa63c73ad4` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:17:32.976902 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5617f0f5-0e55-44ea-878e-2439620cdbc6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5f990fe1a25740f4a655b0fa63c73ad4)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:17:32.978902 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5617f0f5-0e55-44ea-878e-2439620cdbc6 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:17:33.102001 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-5617f0f5-0e55-44ea-878e-2439620cdbc6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5f990fe1a25740f4a655b0fa63c73ad4', 'request_id': 'req-5617f0f5-0e55-44ea-878e-2439620cdbc6'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:17:33.112081 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-5617f0f5-0e55-44ea-878e-2439620cdbc6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5f990fe1a25740f4a655b0fa63c73ad4', 'request_id': 'req-5617f0f5-0e55-44ea-878e-2439620cdbc6'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:17:33.121289 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-5617f0f5-0e55-44ea-878e-2439620cdbc6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5f990fe1a25740f4a655b0fa63c73ad4', 'request_id': 'req-5617f0f5-0e55-44ea-878e-2439620cdbc6'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:17:33.146253 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-5617f0f5-0e55-44ea-878e-2439620cdbc6 admin admin] The token cache is being invalidate because project 5f990fe1a25740f4a655b0fa63c73ad4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 15:17:33.146675 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-5617f0f5-0e55-44ea-878e-2439620cdbc6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:17:33.148376 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-5617f0f5-0e55-44ea-878e-2439620cdbc6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:17:33.153674 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1362/2743] 10.222.0.46 () {58 vars in 1232 bytes} [Thu May 26 15:17:32 2022] DELETE /identity/v3/projects/5f990fe1a25740f4a655b0fa63c73ad4 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:17:33.155481 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b237ec38-08be-4ee2-805b-2ce09bbd64f0 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:17:33.155946 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b237ec38-08be-4ee2-805b-2ce09bbd64f0 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:17:33.156051 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b237ec38-08be-4ee2-805b-2ce09bbd64f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:17:33.182168 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-b237ec38-08be-4ee2-805b-2ce09bbd64f0 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:17:33.183421 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-b237ec38-08be-4ee2-805b-2ce09bbd64f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:17:33.301312 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1382/2744] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:17:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:17:33.309184 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dc3af8-d116-427e-9168-438123a4e3e2 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:17:33.309938 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dc3af8-d116-427e-9168-438123a4e3e2 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:17:33.310349 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48dc3af8-d116-427e-9168-438123a4e3e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:17:33.348729 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-48dc3af8-d116-427e-9168-438123a4e3e2 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:17:33.350947 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-48dc3af8-d116-427e-9168-438123a4e3e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:17:33.362226 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1363/2745] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:17:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:17:33.367546 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b237ec38-08be-4ee2-805b-2ce09bbd64f0 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:17:33.397646 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b237ec38-08be-4ee2-805b-2ce09bbd64f0 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:17:33.407035 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67c2315-ade0-4563-a036-c4b8040c3836 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:17:33.407878 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67c2315-ade0-4563-a036-c4b8040c3836 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:17:33.408264 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67c2315-ade0-4563-a036-c4b8040c3836 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:17:33.409137 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67c2315-ade0-4563-a036-c4b8040c3836 admin admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:17:33.410036 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67c2315-ade0-4563-a036-c4b8040c3836 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 15:17:33.410459 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67c2315-ade0-4563-a036-c4b8040c3836 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:17:33.413500 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67c2315-ade0-4563-a036-c4b8040c3836 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:17:33.434359 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48dc3af8-d116-427e-9168-438123a4e3e2 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:17:33.434624 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1383/2746] 10.222.0.46 () {58 vars in 1152 bytes} [Thu May 26 15:17:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 15:17:33.446692 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48dc3af8-d116-427e-9168-438123a4e3e2 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:17:33.456697 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6849d3df-04de-4a75-a2d5-730e56449129 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:17:33.457605 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6849d3df-04de-4a75-a2d5-730e56449129 admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:17:33.458023 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6849d3df-04de-4a75-a2d5-730e56449129 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:17:33.458406 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6849d3df-04de-4a75-a2d5-730e56449129 admin admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:17:33.461023 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6849d3df-04de-4a75-a2d5-730e56449129 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:17:33.465802 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6849d3df-04de-4a75-a2d5-730e56449129 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:17:33.526901 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1364/2747] 10.222.0.46 () {60 vars in 1152 bytes} [Thu May 26 15:17:33 2022] POST /identity/v3/projects => generated 369 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 15:17:33.528260 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67c2315-ade0-4563-a036-c4b8040c3836 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:17:33.541782 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67c2315-ade0-4563-a036-c4b8040c3836 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:17:33.769841 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a361e3-739a-4296-890c-98fc96e5166f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:17:33.774736 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a361e3-739a-4296-890c-98fc96e5166f admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:17:33.777198 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a361e3-739a-4296-890c-98fc96e5166f admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:17:33.778050 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a361e3-739a-4296-890c-98fc96e5166f admin admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:17:33.782581 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a361e3-739a-4296-890c-98fc96e5166f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:17:33.789822 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8a361e3-739a-4296-890c-98fc96e5166f admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:17:33.875458 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6849d3df-04de-4a75-a2d5-730e56449129 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:17:33.888901 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1384/2748] 10.222.0.46 () {60 vars in 1143 bytes} [Thu May 26 15:17:33 2022] POST /identity/v3/users => generated 285 bytes in 347 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 15:17:33.912172 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6849d3df-04de-4a75-a2d5-730e56449129 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:17:33.930098 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7328ef-e2b2-4751-9138-1f346de51f2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:17:33.931025 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7328ef-e2b2-4751-9138-1f346de51f2a admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:17:33.931332 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7328ef-e2b2-4751-9138-1f346de51f2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:17:33.931795 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7328ef-e2b2-4751-9138-1f346de51f2a admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:17:33.932963 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7328ef-e2b2-4751-9138-1f346de51f2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:17:33.936235 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7328ef-e2b2-4751-9138-1f346de51f2a admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:17:33.951958 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1365/2749] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 15:17:33 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 15:17:33.956296 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a361e3-739a-4296-890c-98fc96e5166f admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:17:33.976059 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a361e3-739a-4296-890c-98fc96e5166f admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:17:33.982866 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af419c07-6b47-4d1f-83bc-c65cfba3918e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:17:33.983566 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af419c07-6b47-4d1f-83bc-c65cfba3918e admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:17:33.983856 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af419c07-6b47-4d1f-83bc-c65cfba3918e admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:17:33.984144 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af419c07-6b47-4d1f-83bc-c65cfba3918e admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:17:33.984907 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af419c07-6b47-4d1f-83bc-c65cfba3918e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:17:33.987319 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af419c07-6b47-4d1f-83bc-c65cfba3918e admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:17:34.007552 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1385/2750] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 15:17:33 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 15:17:34.009360 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7328ef-e2b2-4751-9138-1f346de51f2a admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:17:34.028801 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7328ef-e2b2-4751-9138-1f346de51f2a admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:17:34.040424 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7f1b1d-a633-477b-a45f-2d00bbe4c85e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:17:34.041844 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7f1b1d-a633-477b-a45f-2d00bbe4c85e admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:17:34.042127 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7f1b1d-a633-477b-a45f-2d00bbe4c85e admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:17:34.042405 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7f1b1d-a633-477b-a45f-2d00bbe4c85e admin admin] PATH_INFO: `/v3/projects/38a3d9b61c5841d4b36997b8fb80d206/users/cebde89f2ba541739b98caa0bf0e19e3/roles/d4088d1845a24fbd9bd8975ce0b1d3d0` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:17:34.082560 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7f1b1d-a633-477b-a45f-2d00bbe4c85e admin admin] RBAC: Authorizing `identity:create_grant(project_id=38a3d9b61c5841d4b36997b8fb80d206, user_id=cebde89f2ba541739b98caa0bf0e19e3, role_id=d4088d1845a24fbd9bd8975ce0b1d3d0)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:17:34.086148 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e7f1b1d-a633-477b-a45f-2d00bbe4c85e admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:17:34.131879 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1366/2751] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 15:17:34 2022] PUT /identity/v3/projects/38a3d9b61c5841d4b36997b8fb80d206/users/cebde89f2ba541739b98caa0bf0e19e3/roles/d4088d1845a24fbd9bd8975ce0b1d3d0 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 15:17:34.155253 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a306f788-91fb-428b-b0a4-66a3a591f8a8 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:17:34.156584 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a306f788-91fb-428b-b0a4-66a3a591f8a8 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:17:34.157544 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a306f788-91fb-428b-b0a4-66a3a591f8a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:17:34.171252 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:17:34.221393 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-a306f788-91fb-428b-b0a4-66a3a591f8a8 None None] MFA Rules not processed for user `cebde89f2ba541739b98caa0bf0e19e3`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:17:34.223272 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-a306f788-91fb-428b-b0a4-66a3a591f8a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:17:34.395311 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1386/2752] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:17:34 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:17:34.402222 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7f1b1d-a633-477b-a45f-2d00bbe4c85e admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:17:34.499869 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:17:34.557694 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7f1b1d-a633-477b-a45f-2d00bbe4c85e admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:17:34.582571 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5179a4c6-b5a4-48df-a816-e58a30e15aea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:17:34.583835 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5179a4c6-b5a4-48df-a816-e58a30e15aea admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:17:34.584836 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5179a4c6-b5a4-48df-a816-e58a30e15aea admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:17:34.585333 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5179a4c6-b5a4-48df-a816-e58a30e15aea admin admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:17:34.588009 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5179a4c6-b5a4-48df-a816-e58a30e15aea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:17:34.592796 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5179a4c6-b5a4-48df-a816-e58a30e15aea admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:17:34.686031 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1367/2753] 10.222.0.46 () {60 vars in 1152 bytes} [Thu May 26 15:17:34 2022] POST /identity/v3/projects => generated 369 bytes in 285 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 15:17:34.690994 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a306f788-91fb-428b-b0a4-66a3a591f8a8 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:17:34.719289 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a306f788-91fb-428b-b0a4-66a3a591f8a8 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:17:34.834095 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe5c044-b7a1-40cb-84a7-bd172951b97e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:17:34.835528 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe5c044-b7a1-40cb-84a7-bd172951b97e admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:17:34.835908 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe5c044-b7a1-40cb-84a7-bd172951b97e admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:17:34.836261 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe5c044-b7a1-40cb-84a7-bd172951b97e admin admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:17:34.837046 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe5c044-b7a1-40cb-84a7-bd172951b97e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:17:34.839898 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbe5c044-b7a1-40cb-84a7-bd172951b97e admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:17:34.902658 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1387/2754] 10.222.0.46 () {60 vars in 1143 bytes} [Thu May 26 15:17:34 2022] POST /identity/v3/users => generated 291 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 15:17:34.908005 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5179a4c6-b5a4-48df-a816-e58a30e15aea admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:17:34.953040 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5179a4c6-b5a4-48df-a816-e58a30e15aea admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:17:34.970459 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304589c6-0241-44c6-8bf1-87b056488425 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:17:34.971692 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304589c6-0241-44c6-8bf1-87b056488425 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:17:34.972474 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304589c6-0241-44c6-8bf1-87b056488425 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:17:34.973167 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304589c6-0241-44c6-8bf1-87b056488425 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:17:34.974950 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304589c6-0241-44c6-8bf1-87b056488425 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:17:34.977984 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304589c6-0241-44c6-8bf1-87b056488425 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:17:35.002771 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1368/2755] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 15:17:34 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 15:17:35.007546 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe5c044-b7a1-40cb-84a7-bd172951b97e admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:17:35.025128 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbe5c044-b7a1-40cb-84a7-bd172951b97e admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:17:35.039375 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c7d15a-bbff-4627-9102-16e7775352ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:17:35.040296 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c7d15a-bbff-4627-9102-16e7775352ea admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:17:35.040578 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c7d15a-bbff-4627-9102-16e7775352ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:17:35.040856 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c7d15a-bbff-4627-9102-16e7775352ea admin admin] PATH_INFO: `/v3/projects/108f85ce32ad4a1680e4e4a03e6a1766/users/db1589ffb29e459c957651b4ba433abf/roles/a2e551773cbd4df084020676865b8222` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:17:35.068049 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c7d15a-bbff-4627-9102-16e7775352ea admin admin] RBAC: Authorizing `identity:create_grant(project_id=108f85ce32ad4a1680e4e4a03e6a1766, user_id=db1589ffb29e459c957651b4ba433abf, role_id=a2e551773cbd4df084020676865b8222)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:17:35.071489 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c7d15a-bbff-4627-9102-16e7775352ea admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:17:35.117511 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1388/2756] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 15:17:35 2022] PUT /identity/v3/projects/108f85ce32ad4a1680e4e4a03e6a1766/users/db1589ffb29e459c957651b4ba433abf/roles/a2e551773cbd4df084020676865b8222 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:17:35.122499 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304589c6-0241-44c6-8bf1-87b056488425 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:17:35.158650 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304589c6-0241-44c6-8bf1-87b056488425 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:17:35.284451 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded7ea66-5dba-4cf2-838d-32e34f70f65e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:17:35.285547 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded7ea66-5dba-4cf2-838d-32e34f70f65e admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:17:35.286082 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded7ea66-5dba-4cf2-838d-32e34f70f65e admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:17:35.286864 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded7ea66-5dba-4cf2-838d-32e34f70f65e admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:17:35.288766 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded7ea66-5dba-4cf2-838d-32e34f70f65e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:17:35.291553 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded7ea66-5dba-4cf2-838d-32e34f70f65e admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:17:35.310323 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1369/2757] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 15:17:35 2022] GET /identity/v3/roles => generated 1420 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 15:17:35.314492 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c7d15a-bbff-4627-9102-16e7775352ea admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:17:35.344014 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c7d15a-bbff-4627-9102-16e7775352ea admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:17:35.352034 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3c9aa8-b96b-46f2-b805-fdfacce2b1f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:17:35.353436 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3c9aa8-b96b-46f2-b805-fdfacce2b1f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:17:35.353436 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3c9aa8-b96b-46f2-b805-fdfacce2b1f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:17:35.353949 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3c9aa8-b96b-46f2-b805-fdfacce2b1f7 admin admin] PATH_INFO: `/v3/projects/108f85ce32ad4a1680e4e4a03e6a1766/users/db1589ffb29e459c957651b4ba433abf/roles/a2e551773cbd4df084020676865b8222` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:17:35.356916 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3c9aa8-b96b-46f2-b805-fdfacce2b1f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=108f85ce32ad4a1680e4e4a03e6a1766, user_id=db1589ffb29e459c957651b4ba433abf, role_id=a2e551773cbd4df084020676865b8222)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:17:35.360632 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3c9aa8-b96b-46f2-b805-fdfacce2b1f7 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:17:35.377725 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1389/2758] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 15:17:35 2022] PUT /identity/v3/projects/108f85ce32ad4a1680e4e4a03e6a1766/users/db1589ffb29e459c957651b4ba433abf/roles/a2e551773cbd4df084020676865b8222 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:17:35.396003 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbbaab8-47dd-4b38-9d14-70fc3f5da58d None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:17:35.396298 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbbaab8-47dd-4b38-9d14-70fc3f5da58d None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:17:35.396576 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbbaab8-47dd-4b38-9d14-70fc3f5da58d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:17:35.472730 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-ddbbaab8-47dd-4b38-9d14-70fc3f5da58d None None] MFA Rules not processed for user `db1589ffb29e459c957651b4ba433abf`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:17:35.476079 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-ddbbaab8-47dd-4b38-9d14-70fc3f5da58d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:17:35.689197 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1370/2759] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:17:35 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 295 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:17:35.716617 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3c9aa8-b96b-46f2-b805-fdfacce2b1f7 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:17:35.718909 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-ed3c9aa8-b96b-46f2-b805-fdfacce2b1f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:17:35.898504 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:17:35.915329 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3c9aa8-b96b-46f2-b805-fdfacce2b1f7 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:17:35.923116 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab14e7c-cc6c-4e9b-bce3-450a20837b60 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5b5b35614a547d783e6e77ddb8c1c03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:17:35.925713 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab14e7c-cc6c-4e9b-bce3-450a20837b60 service cinder] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:17:35.926141 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab14e7c-cc6c-4e9b-bce3-450a20837b60 service cinder] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:17:35.927478 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab14e7c-cc6c-4e9b-bce3-450a20837b60 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:17:35.950963 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab14e7c-cc6c-4e9b-bce3-450a20837b60 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:17:35.952947 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab14e7c-cc6c-4e9b-bce3-450a20837b60 service cinder] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:17:35.969173 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1390/2760] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:17:35 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 253 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 15:17:36.168715 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531d62d0-d341-4dc8-ac09-3f1fc72f065b None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:17:36.169163 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531d62d0-d341-4dc8-ac09-3f1fc72f065b None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:17:36.169511 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531d62d0-d341-4dc8-ac09-3f1fc72f065b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:17:36.219551 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-531d62d0-d341-4dc8-ac09-3f1fc72f065b None None] MFA Rules not processed for user `176fec434836494a9af12c96be3a8eb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:17:36.221209 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-531d62d0-d341-4dc8-ac09-3f1fc72f065b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:17:36.251374 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:17:36.350784 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1371/2761] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:17:36 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:17:36.356624 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab14e7c-cc6c-4e9b-bce3-450a20837b60 service cinder] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:17:36.360450 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-3ab14e7c-cc6c-4e9b-bce3-450a20837b60 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:17:36.533035 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab14e7c-cc6c-4e9b-bce3-450a20837b60 service cinder] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:17:36.563875 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3acd416-0d4e-41d1-aee7-22a4797de9e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:17:36.564606 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3acd416-0d4e-41d1-aee7-22a4797de9e1 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:17:36.564891 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3acd416-0d4e-41d1-aee7-22a4797de9e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:17:36.565171 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3acd416-0d4e-41d1-aee7-22a4797de9e1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:17:36.565929 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3acd416-0d4e-41d1-aee7-22a4797de9e1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:17:36.567904 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3acd416-0d4e-41d1-aee7-22a4797de9e1 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:17:36.623108 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1391/2762] 10.222.0.46 () {60 vars in 1152 bytes} [Thu May 26 15:17:36 2022] POST /identity/v3/projects => generated 379 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 15:17:36.627717 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531d62d0-d341-4dc8-ac09-3f1fc72f065b None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:17:36.649231 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531d62d0-d341-4dc8-ac09-3f1fc72f065b None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:17:36.747842 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c638f41-2832-4c8e-9436-920ca34e98c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:17:36.748563 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c638f41-2832-4c8e-9436-920ca34e98c4 admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:17:36.748850 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c638f41-2832-4c8e-9436-920ca34e98c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:17:36.749130 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c638f41-2832-4c8e-9436-920ca34e98c4 admin admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:17:36.750828 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c638f41-2832-4c8e-9436-920ca34e98c4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:17:36.752127 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c638f41-2832-4c8e-9436-920ca34e98c4 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:17:36.782999 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1372/2763] 10.222.0.46 () {60 vars in 1143 bytes} [Thu May 26 15:17:36 2022] POST /identity/v3/users => generated 296 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 15:17:36.784433 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3acd416-0d4e-41d1-aee7-22a4797de9e1 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:17:36.801237 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3acd416-0d4e-41d1-aee7-22a4797de9e1 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:17:36.809742 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89497b0d-9128-4f08-a1e0-0d897845591d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:17:36.811233 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89497b0d-9128-4f08-a1e0-0d897845591d admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:17:36.812376 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89497b0d-9128-4f08-a1e0-0d897845591d admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:17:36.812788 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89497b0d-9128-4f08-a1e0-0d897845591d admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:17:36.813678 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89497b0d-9128-4f08-a1e0-0d897845591d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:17:36.817910 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89497b0d-9128-4f08-a1e0-0d897845591d admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:17:36.847990 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1392/2764] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 15:17:36 2022] GET /identity/v3/roles => generated 1420 bytes in 64 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 15:17:36.852937 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c638f41-2832-4c8e-9436-920ca34e98c4 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:17:36.875085 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c638f41-2832-4c8e-9436-920ca34e98c4 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:17:36.882745 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005b1012-a50a-4b27-9864-6efb98a35345 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:17:36.883850 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005b1012-a50a-4b27-9864-6efb98a35345 admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:17:36.884112 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005b1012-a50a-4b27-9864-6efb98a35345 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:17:36.884482 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005b1012-a50a-4b27-9864-6efb98a35345 admin admin] PATH_INFO: `/v3/projects/4658c39721b44ff9a57416f948913eb6/users/2991d18c077c470eab993644b445bce9/roles/a2e551773cbd4df084020676865b8222` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:17:36.904307 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005b1012-a50a-4b27-9864-6efb98a35345 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4658c39721b44ff9a57416f948913eb6, user_id=2991d18c077c470eab993644b445bce9, role_id=a2e551773cbd4df084020676865b8222)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:17:36.908432 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005b1012-a50a-4b27-9864-6efb98a35345 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:17:36.940913 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1373/2765] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 15:17:36 2022] PUT /identity/v3/projects/4658c39721b44ff9a57416f948913eb6/users/2991d18c077c470eab993644b445bce9/roles/a2e551773cbd4df084020676865b8222 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 15:17:36.942130 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89497b0d-9128-4f08-a1e0-0d897845591d admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:17:36.959608 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89497b0d-9128-4f08-a1e0-0d897845591d admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:17:37.050327 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663fe480-480e-462f-9741-5f7511626201 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:17:37.050327 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663fe480-480e-462f-9741-5f7511626201 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:17:37.050929 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663fe480-480e-462f-9741-5f7511626201 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:17:37.051300 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663fe480-480e-462f-9741-5f7511626201 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:17:37.052182 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663fe480-480e-462f-9741-5f7511626201 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:17:37.054698 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663fe480-480e-462f-9741-5f7511626201 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:17:37.074760 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1393/2766] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 15:17:36 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 15:17:37.078585 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005b1012-a50a-4b27-9864-6efb98a35345 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:17:37.102849 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005b1012-a50a-4b27-9864-6efb98a35345 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:17:37.113613 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4816ab90-b203-47cd-a345-c17f5a0c23bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:17:37.114684 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4816ab90-b203-47cd-a345-c17f5a0c23bd admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:17:37.114980 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4816ab90-b203-47cd-a345-c17f5a0c23bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:17:37.115591 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4816ab90-b203-47cd-a345-c17f5a0c23bd admin admin] PATH_INFO: `/v3/projects/4658c39721b44ff9a57416f948913eb6/users/2991d18c077c470eab993644b445bce9/roles/a2e551773cbd4df084020676865b8222` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:17:37.117871 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4816ab90-b203-47cd-a345-c17f5a0c23bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=4658c39721b44ff9a57416f948913eb6, user_id=2991d18c077c470eab993644b445bce9, role_id=a2e551773cbd4df084020676865b8222)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:17:37.120638 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4816ab90-b203-47cd-a345-c17f5a0c23bd admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:17:37.145810 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1374/2767] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 15:17:37 2022] PUT /identity/v3/projects/4658c39721b44ff9a57416f948913eb6/users/2991d18c077c470eab993644b445bce9/roles/a2e551773cbd4df084020676865b8222 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:17:38.344490 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663fe480-480e-462f-9741-5f7511626201 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:17:38.360036 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663fe480-480e-462f-9741-5f7511626201 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:17:38.466993 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78026ca3-a75b-4aa9-aff3-8fffe21f17ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:17:38.468152 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78026ca3-a75b-4aa9-aff3-8fffe21f17ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:17:38.468770 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78026ca3-a75b-4aa9-aff3-8fffe21f17ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:17:38.469275 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78026ca3-a75b-4aa9-aff3-8fffe21f17ab admin admin] PATH_INFO: `/v3/users/cebde89f2ba541739b98caa0bf0e19e3` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:17:38.471147 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78026ca3-a75b-4aa9-aff3-8fffe21f17ab admin admin] RBAC: Authorizing `identity:delete_user(user_id=cebde89f2ba541739b98caa0bf0e19e3)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:17:38.473063 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78026ca3-a75b-4aa9-aff3-8fffe21f17ab admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:17:38.475708 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:17:38.602076 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-78026ca3-a75b-4aa9-aff3-8fffe21f17ab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cebde89f2ba541739b98caa0bf0e19e3'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:17:38.617134 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-78026ca3-a75b-4aa9-aff3-8fffe21f17ab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cebde89f2ba541739b98caa0bf0e19e3'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:17:38.619023 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-78026ca3-a75b-4aa9-aff3-8fffe21f17ab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cebde89f2ba541739b98caa0bf0e19e3'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:17:38.635913 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-78026ca3-a75b-4aa9-aff3-8fffe21f17ab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cebde89f2ba541739b98caa0bf0e19e3'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:17:38.648389 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-78026ca3-a75b-4aa9-aff3-8fffe21f17ab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cebde89f2ba541739b98caa0bf0e19e3'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:17:38.658025 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1394/2768] 10.222.0.46 () {58 vars in 1223 bytes} [Thu May 26 15:17:38 2022] DELETE /identity/v3/users/cebde89f2ba541739b98caa0bf0e19e3 => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:17:38.662575 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d57ffc-0967-440d-8038-62e43969187b None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:17:38.663076 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d57ffc-0967-440d-8038-62e43969187b None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:17:38.663198 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d57ffc-0967-440d-8038-62e43969187b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:17:38.701221 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-03d57ffc-0967-440d-8038-62e43969187b None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:17:38.703648 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-03d57ffc-0967-440d-8038-62e43969187b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:17:38.794670 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1375/2769] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:17:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:17:38.809851 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78026ca3-a75b-4aa9-aff3-8fffe21f17ab admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:17:38.812834 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-78026ca3-a75b-4aa9-aff3-8fffe21f17ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:17:38.912777 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78026ca3-a75b-4aa9-aff3-8fffe21f17ab admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:17:38.916980 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad76a242-fa05-4e84-9e4b-be26abe1719d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0156a2f6f3794f479e76b85bdbbe5c26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:17:38.918128 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad76a242-fa05-4e84-9e4b-be26abe1719d service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:17:38.918568 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad76a242-fa05-4e84-9e4b-be26abe1719d service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:17:38.919004 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad76a242-fa05-4e84-9e4b-be26abe1719d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:17:38.936484 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad76a242-fa05-4e84-9e4b-be26abe1719d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:17:38.938110 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad76a242-fa05-4e84-9e4b-be26abe1719d service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:17:38.955593 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1395/2770] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:17:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 15:17:39.382608 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d57ffc-0967-440d-8038-62e43969187b None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:17:39.384176 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-03d57ffc-0967-440d-8038-62e43969187b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:17:39.410043 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d57ffc-0967-440d-8038-62e43969187b None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:17:39.416571 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22aedaf-204a-4bf5-a4a9-cb301bbad0f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:17:39.417471 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22aedaf-204a-4bf5-a4a9-cb301bbad0f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:17:39.417800 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22aedaf-204a-4bf5-a4a9-cb301bbad0f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:17:39.418072 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22aedaf-204a-4bf5-a4a9-cb301bbad0f5 admin admin] PATH_INFO: `/v3/projects/38a3d9b61c5841d4b36997b8fb80d206` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:17:39.419340 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22aedaf-204a-4bf5-a4a9-cb301bbad0f5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=38a3d9b61c5841d4b36997b8fb80d206)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:17:39.421536 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22aedaf-204a-4bf5-a4a9-cb301bbad0f5 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:17:39.430221 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:17:39.549111 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-d22aedaf-204a-4bf5-a4a9-cb301bbad0f5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '38a3d9b61c5841d4b36997b8fb80d206', 'request_id': 'req-d22aedaf-204a-4bf5-a4a9-cb301bbad0f5'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:17:39.565824 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-d22aedaf-204a-4bf5-a4a9-cb301bbad0f5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '38a3d9b61c5841d4b36997b8fb80d206', 'request_id': 'req-d22aedaf-204a-4bf5-a4a9-cb301bbad0f5'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:17:39.577126 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-d22aedaf-204a-4bf5-a4a9-cb301bbad0f5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '38a3d9b61c5841d4b36997b8fb80d206', 'request_id': 'req-d22aedaf-204a-4bf5-a4a9-cb301bbad0f5'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:17:39.616319 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-d22aedaf-204a-4bf5-a4a9-cb301bbad0f5 admin admin] The token cache is being invalidate because project 38a3d9b61c5841d4b36997b8fb80d206 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 15:17:39.616731 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-d22aedaf-204a-4bf5-a4a9-cb301bbad0f5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:17:39.617628 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-d22aedaf-204a-4bf5-a4a9-cb301bbad0f5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:17:39.638510 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1376/2771] 10.222.0.46 () {58 vars in 1232 bytes} [Thu May 26 15:17:39 2022] DELETE /identity/v3/projects/38a3d9b61c5841d4b36997b8fb80d206 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:17:39.642480 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad76a242-fa05-4e84-9e4b-be26abe1719d service neutron] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:17:39.685746 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad76a242-fa05-4e84-9e4b-be26abe1719d service neutron] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:17:39.783661 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47382b11-f66f-462f-8196-872c0513ace4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:17:39.785010 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47382b11-f66f-462f-8196-872c0513ace4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:17:39.785521 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47382b11-f66f-462f-8196-872c0513ace4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:17:39.785975 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47382b11-f66f-462f-8196-872c0513ace4 admin admin] PATH_INFO: `/v3/users/db1589ffb29e459c957651b4ba433abf` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:17:39.803130 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47382b11-f66f-462f-8196-872c0513ace4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=db1589ffb29e459c957651b4ba433abf)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:17:39.808046 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47382b11-f66f-462f-8196-872c0513ace4 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:17:39.872945 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:17:39.930173 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-47382b11-f66f-462f-8196-872c0513ace4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'db1589ffb29e459c957651b4ba433abf'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:17:39.941809 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-47382b11-f66f-462f-8196-872c0513ace4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'db1589ffb29e459c957651b4ba433abf'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:17:39.942412 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-47382b11-f66f-462f-8196-872c0513ace4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'db1589ffb29e459c957651b4ba433abf'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:17:39.957411 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-47382b11-f66f-462f-8196-872c0513ace4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'db1589ffb29e459c957651b4ba433abf'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:17:39.968914 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-47382b11-f66f-462f-8196-872c0513ace4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'db1589ffb29e459c957651b4ba433abf'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:17:39.981255 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1396/2772] 10.222.0.46 () {58 vars in 1223 bytes} [Thu May 26 15:17:39 2022] DELETE /identity/v3/users/db1589ffb29e459c957651b4ba433abf => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:17:40.410640 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22aedaf-204a-4bf5-a4a9-cb301bbad0f5 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:17:40.412792 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-d22aedaf-204a-4bf5-a4a9-cb301bbad0f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:17:40.457355 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22aedaf-204a-4bf5-a4a9-cb301bbad0f5 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:17:40.601954 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da0bd68-43af-4c68-a1a0-44bdbe718045 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:17:40.603000 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da0bd68-43af-4c68-a1a0-44bdbe718045 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:17:40.603285 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da0bd68-43af-4c68-a1a0-44bdbe718045 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:17:40.603614 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da0bd68-43af-4c68-a1a0-44bdbe718045 admin admin] PATH_INFO: `/v3/projects/108f85ce32ad4a1680e4e4a03e6a1766` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:17:40.611024 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da0bd68-43af-4c68-a1a0-44bdbe718045 admin admin] RBAC: Authorizing `identity:delete_project(project_id=108f85ce32ad4a1680e4e4a03e6a1766)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:17:40.611024 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da0bd68-43af-4c68-a1a0-44bdbe718045 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:17:40.705431 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47382b11-f66f-462f-8196-872c0513ace4 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:17:40.707627 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-47382b11-f66f-462f-8196-872c0513ace4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:17:40.729891 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47382b11-f66f-462f-8196-872c0513ace4 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:17:40.731232 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-4da0bd68-43af-4c68-a1a0-44bdbe718045 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '108f85ce32ad4a1680e4e4a03e6a1766', 'request_id': 'req-4da0bd68-43af-4c68-a1a0-44bdbe718045'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:17:40.743839 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-4da0bd68-43af-4c68-a1a0-44bdbe718045 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '108f85ce32ad4a1680e4e4a03e6a1766', 'request_id': 'req-4da0bd68-43af-4c68-a1a0-44bdbe718045'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:17:40.752746 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-4da0bd68-43af-4c68-a1a0-44bdbe718045 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '108f85ce32ad4a1680e4e4a03e6a1766', 'request_id': 'req-4da0bd68-43af-4c68-a1a0-44bdbe718045'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:17:40.778109 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-4da0bd68-43af-4c68-a1a0-44bdbe718045 admin admin] The token cache is being invalidate because project 108f85ce32ad4a1680e4e4a03e6a1766 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 15:17:40.778740 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-4da0bd68-43af-4c68-a1a0-44bdbe718045 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:17:40.780535 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-4da0bd68-43af-4c68-a1a0-44bdbe718045 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:17:40.787016 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1377/2773] 10.222.0.46 () {58 vars in 1232 bytes} [Thu May 26 15:17:40 2022] DELETE /identity/v3/projects/108f85ce32ad4a1680e4e4a03e6a1766 => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:17:40.788897 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71cc254-dec5-4c5e-adb9-3735912d0450 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:17:40.789188 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71cc254-dec5-4c5e-adb9-3735912d0450 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:17:40.789494 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d71cc254-dec5-4c5e-adb9-3735912d0450 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:17:40.860260 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-d71cc254-dec5-4c5e-adb9-3735912d0450 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:17:40.862219 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-d71cc254-dec5-4c5e-adb9-3735912d0450 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:17:40.913095 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55aa6f6-eec6-4fa2-8b48-afdd7155a913 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:17:40.915256 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55aa6f6-eec6-4fa2-8b48-afdd7155a913 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:17:40.915256 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55aa6f6-eec6-4fa2-8b48-afdd7155a913 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:17:40.915256 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55aa6f6-eec6-4fa2-8b48-afdd7155a913 admin admin] PATH_INFO: `/v3/users/993290e392434879a2a9a9c283b2caad` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:17:40.919087 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55aa6f6-eec6-4fa2-8b48-afdd7155a913 admin admin] RBAC: Authorizing `identity:delete_user(user_id=993290e392434879a2a9a9c283b2caad)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:17:40.921098 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55aa6f6-eec6-4fa2-8b48-afdd7155a913 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:17:40.923366 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:17:41.001551 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1378/2774] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:17:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:17:41.013976 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0792ae7f-a9ca-4c19-9ec7-d6b420c32c7a None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:17:41.014848 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0792ae7f-a9ca-4c19-9ec7-d6b420c32c7a None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:17:41.015478 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0792ae7f-a9ca-4c19-9ec7-d6b420c32c7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:17:41.020059 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-d55aa6f6-eec6-4fa2-8b48-afdd7155a913 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '993290e392434879a2a9a9c283b2caad'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:17:41.033875 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-d55aa6f6-eec6-4fa2-8b48-afdd7155a913 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '993290e392434879a2a9a9c283b2caad'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:17:41.035392 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-d55aa6f6-eec6-4fa2-8b48-afdd7155a913 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '993290e392434879a2a9a9c283b2caad'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:17:41.049201 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-0792ae7f-a9ca-4c19-9ec7-d6b420c32c7a None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:17:41.051634 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-0792ae7f-a9ca-4c19-9ec7-d6b420c32c7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:17:41.059609 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-d55aa6f6-eec6-4fa2-8b48-afdd7155a913 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '993290e392434879a2a9a9c283b2caad'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:17:41.067676 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1379/2775] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:17:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:17:41.070602 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-d55aa6f6-eec6-4fa2-8b48-afdd7155a913 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '993290e392434879a2a9a9c283b2caad'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:17:41.074330 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0792ae7f-a9ca-4c19-9ec7-d6b420c32c7a None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:17:41.079557 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1397/2776] 10.222.0.46 () {58 vars in 1223 bytes} [Thu May 26 15:17:40 2022] DELETE /identity/v3/users/993290e392434879a2a9a9c283b2caad => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:17:41.084583 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493b9a3d-dcaa-4f6c-85c3-fe75f3907928 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:17:41.084880 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493b9a3d-dcaa-4f6c-85c3-fe75f3907928 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:17:41.085162 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493b9a3d-dcaa-4f6c-85c3-fe75f3907928 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:17:41.095959 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0792ae7f-a9ca-4c19-9ec7-d6b420c32c7a None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:17:41.139647 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-493b9a3d-dcaa-4f6c-85c3-fe75f3907928 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:17:41.143165 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-493b9a3d-dcaa-4f6c-85c3-fe75f3907928 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:17:41.176952 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:17:41.253571 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa0e899-4c56-4c00-9df9-315716526207 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:17:41.254156 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa0e899-4c56-4c00-9df9-315716526207 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:17:41.254438 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa0e899-4c56-4c00-9df9-315716526207 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:17:41.254719 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa0e899-4c56-4c00-9df9-315716526207 admin admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:17:41.256904 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa0e899-4c56-4c00-9df9-315716526207 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 15:17:41.261720 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa0e899-4c56-4c00-9df9-315716526207 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:17:41.261720 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa0e899-4c56-4c00-9df9-315716526207 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:17:41.285519 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1380/2777] 10.222.0.46 () {58 vars in 1152 bytes} [Thu May 26 15:17:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 15:17:41.290257 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa0e899-4c56-4c00-9df9-315716526207 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:17:41.300189 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1398/2778] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:17:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:17:41.302968 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa0e899-4c56-4c00-9df9-315716526207 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:17:41.310222 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264342e1-b1a2-4262-92fd-b4743b0d92df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:17:41.311021 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264342e1-b1a2-4262-92fd-b4743b0d92df admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:17:41.311372 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264342e1-b1a2-4262-92fd-b4743b0d92df admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:17:41.311755 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264342e1-b1a2-4262-92fd-b4743b0d92df admin admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:17:41.312524 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264342e1-b1a2-4262-92fd-b4743b0d92df admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:17:41.314757 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264342e1-b1a2-4262-92fd-b4743b0d92df admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:17:41.315858 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493b9a3d-dcaa-4f6c-85c3-fe75f3907928 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:17:41.323260 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-493b9a3d-dcaa-4f6c-85c3-fe75f3907928 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:17:41.354698 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1381/2779] 10.222.0.46 () {60 vars in 1152 bytes} [Thu May 26 15:17:41 2022] POST /identity/v3/projects => generated 387 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 15:17:41.361352 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264342e1-b1a2-4262-92fd-b4743b0d92df admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:17:41.453537 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493b9a3d-dcaa-4f6c-85c3-fe75f3907928 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:17:41.470266 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264342e1-b1a2-4262-92fd-b4743b0d92df admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:17:41.477215 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e4226d-fd91-4bc5-b0de-e0fc70152d53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:17:41.477578 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e4226d-fd91-4bc5-b0de-e0fc70152d53 admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:17:41.477901 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e4226d-fd91-4bc5-b0de-e0fc70152d53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:17:41.478280 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e4226d-fd91-4bc5-b0de-e0fc70152d53 admin admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:17:41.479109 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e4226d-fd91-4bc5-b0de-e0fc70152d53 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:17:41.481426 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e4226d-fd91-4bc5-b0de-e0fc70152d53 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:17:41.523658 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1382/2780] 10.222.0.46 () {60 vars in 1143 bytes} [Thu May 26 15:17:41 2022] POST /identity/v3/users => generated 294 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 15:17:41.524883 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e4226d-fd91-4bc5-b0de-e0fc70152d53 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:17:41.559070 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e4226d-fd91-4bc5-b0de-e0fc70152d53 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:17:41.564743 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a32af9-299e-44a4-af38-f172bf935d14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:17:41.565470 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a32af9-299e-44a4-af38-f172bf935d14 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:17:41.565923 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a32af9-299e-44a4-af38-f172bf935d14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:17:41.566251 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a32af9-299e-44a4-af38-f172bf935d14 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:17:41.567238 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a32af9-299e-44a4-af38-f172bf935d14 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:17:41.603801 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a32af9-299e-44a4-af38-f172bf935d14 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:17:41.644884 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1383/2781] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 15:17:41 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 15:17:41.652479 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a32af9-299e-44a4-af38-f172bf935d14 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:17:41.691092 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a32af9-299e-44a4-af38-f172bf935d14 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:17:41.712228 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3091ed79-378e-47fb-a886-5a14128da8d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:17:41.716186 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3091ed79-378e-47fb-a886-5a14128da8d4 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:17:41.717611 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3091ed79-378e-47fb-a886-5a14128da8d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:17:41.720079 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3091ed79-378e-47fb-a886-5a14128da8d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:17:41.720549 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7522e17f-dbb5-409a-b710-3e042560d863 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0156a2f6f3794f479e76b85bdbbe5c26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:17:41.722975 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3091ed79-378e-47fb-a886-5a14128da8d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:17:41.739637 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3091ed79-378e-47fb-a886-5a14128da8d4 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:17:41.749802 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7522e17f-dbb5-409a-b710-3e042560d863 service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:17:41.750154 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7522e17f-dbb5-409a-b710-3e042560d863 service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:17:41.750394 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7522e17f-dbb5-409a-b710-3e042560d863 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:17:41.756641 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:17:41.776099 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7522e17f-dbb5-409a-b710-3e042560d863 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:17:41.789480 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7522e17f-dbb5-409a-b710-3e042560d863 service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:17:41.802682 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1384/2782] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 15:17:41 2022] GET /identity/v3/roles => generated 1420 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 15:17:41.807733 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3091ed79-378e-47fb-a886-5a14128da8d4 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:17:41.827114 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1399/2783] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:17:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 512 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 15:17:41.853231 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3091ed79-378e-47fb-a886-5a14128da8d4 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:17:41.861433 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f9b63a-ce35-4784-b2ad-712914265659 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:17:41.863594 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f9b63a-ce35-4784-b2ad-712914265659 admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:17:41.864037 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f9b63a-ce35-4784-b2ad-712914265659 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:17:41.864459 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f9b63a-ce35-4784-b2ad-712914265659 admin admin] PATH_INFO: `/v3/projects/72d3efe0d5d646bbafdaad6598bbacb7/users/4a4e64641afd484f9fc4cf7fc8007ea2/roles/d4088d1845a24fbd9bd8975ce0b1d3d0` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:17:41.913705 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f9b63a-ce35-4784-b2ad-712914265659 admin admin] RBAC: Authorizing `identity:create_grant(project_id=72d3efe0d5d646bbafdaad6598bbacb7, user_id=4a4e64641afd484f9fc4cf7fc8007ea2, role_id=d4088d1845a24fbd9bd8975ce0b1d3d0)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:17:41.933770 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f9b63a-ce35-4784-b2ad-712914265659 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:17:41.977157 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1385/2784] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 15:17:41 2022] PUT /identity/v3/projects/72d3efe0d5d646bbafdaad6598bbacb7/users/4a4e64641afd484f9fc4cf7fc8007ea2/roles/d4088d1845a24fbd9bd8975ce0b1d3d0 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:17:41.977688 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1735deae-88c6-48de-866c-05e21d2ed4a1 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:17:41.978167 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1735deae-88c6-48de-866c-05e21d2ed4a1 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:17:41.978666 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1735deae-88c6-48de-866c-05e21d2ed4a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:17:42.053235 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-1735deae-88c6-48de-866c-05e21d2ed4a1 None None] MFA Rules not processed for user `4a4e64641afd484f9fc4cf7fc8007ea2`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:17:42.054504 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-1735deae-88c6-48de-866c-05e21d2ed4a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:17:42.188236 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1400/2785] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:17:41 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:17:42.190177 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f9b63a-ce35-4784-b2ad-712914265659 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:17:42.310162 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f9b63a-ce35-4784-b2ad-712914265659 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:17:42.321800 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d26b94-3ebb-40c8-a5ae-832057654d6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:17:42.322729 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d26b94-3ebb-40c8-a5ae-832057654d6c admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:17:42.323022 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d26b94-3ebb-40c8-a5ae-832057654d6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:17:42.323431 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d26b94-3ebb-40c8-a5ae-832057654d6c admin admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:17:42.324433 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d26b94-3ebb-40c8-a5ae-832057654d6c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:17:42.338377 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d26b94-3ebb-40c8-a5ae-832057654d6c admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:17:42.439675 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1386/2786] 10.222.0.46 () {60 vars in 1152 bytes} [Thu May 26 15:17:42 2022] POST /identity/v3/projects => generated 387 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 15:17:42.442059 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1735deae-88c6-48de-866c-05e21d2ed4a1 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:17:42.447791 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d26b94-3ebb-40c8-a5ae-832057654d6c admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:17:42.454426 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-52d26b94-3ebb-40c8-a5ae-832057654d6c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:17:42.484461 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1735deae-88c6-48de-866c-05e21d2ed4a1 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:17:42.496287 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:17:42.603510 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c0f515-c3c3-4b6a-91ec-b5db6d740f90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:17:42.604733 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c0f515-c3c3-4b6a-91ec-b5db6d740f90 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:17:42.605015 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c0f515-c3c3-4b6a-91ec-b5db6d740f90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:17:42.605292 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c0f515-c3c3-4b6a-91ec-b5db6d740f90 admin admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:17:42.606331 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c0f515-c3c3-4b6a-91ec-b5db6d740f90 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:17:42.609242 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c0f515-c3c3-4b6a-91ec-b5db6d740f90 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:17:42.661532 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1401/2787] 10.222.0.46 () {60 vars in 1143 bytes} [Thu May 26 15:17:42 2022] POST /identity/v3/users => generated 300 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 15:17:42.662578 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d26b94-3ebb-40c8-a5ae-832057654d6c admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:17:42.667412 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c0f515-c3c3-4b6a-91ec-b5db6d740f90 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:17:42.670197 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e39e94-925d-42dd-afd7-4e0ccb329284 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:17:42.671580 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e39e94-925d-42dd-afd7-4e0ccb329284 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:17:42.672141 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e39e94-925d-42dd-afd7-4e0ccb329284 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:17:42.672724 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e39e94-925d-42dd-afd7-4e0ccb329284 admin admin] PATH_INFO: `/v3/projects/78466c91318f41baa08bff10235adb75` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:17:42.674979 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e39e94-925d-42dd-afd7-4e0ccb329284 admin admin] RBAC: Authorizing `identity:delete_project(project_id=78466c91318f41baa08bff10235adb75)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:17:42.679300 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e39e94-925d-42dd-afd7-4e0ccb329284 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:17:42.698551 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c0f515-c3c3-4b6a-91ec-b5db6d740f90 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:17:42.712548 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a6500c-d97f-45a9-8b21-47cc3a5960b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:17:42.713706 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a6500c-d97f-45a9-8b21-47cc3a5960b8 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:17:42.714414 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a6500c-d97f-45a9-8b21-47cc3a5960b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:17:42.714992 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a6500c-d97f-45a9-8b21-47cc3a5960b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:17:42.716633 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a6500c-d97f-45a9-8b21-47cc3a5960b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:17:42.720080 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a6500c-d97f-45a9-8b21-47cc3a5960b8 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:17:42.741252 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1402/2788] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 15:17:42 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 15:17:42.748198 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a6500c-d97f-45a9-8b21-47cc3a5960b8 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:17:42.781439 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a6500c-d97f-45a9-8b21-47cc3a5960b8 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:17:42.788748 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-44e39e94-925d-42dd-afd7-4e0ccb329284 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '78466c91318f41baa08bff10235adb75', 'request_id': 'req-44e39e94-925d-42dd-afd7-4e0ccb329284'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:17:42.807188 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-44e39e94-925d-42dd-afd7-4e0ccb329284 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '78466c91318f41baa08bff10235adb75', 'request_id': 'req-44e39e94-925d-42dd-afd7-4e0ccb329284'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:17:42.838520 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=92,keystone:INSERT=2,keystone:DELETE=6 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:17:42.842977 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-44e39e94-925d-42dd-afd7-4e0ccb329284 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '78466c91318f41baa08bff10235adb75', 'request_id': 'req-44e39e94-925d-42dd-afd7-4e0ccb329284'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:17:42.879538 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-44e39e94-925d-42dd-afd7-4e0ccb329284 admin admin] The token cache is being invalidate because project 78466c91318f41baa08bff10235adb75 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 15:17:42.880047 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-44e39e94-925d-42dd-afd7-4e0ccb329284 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:17:42.883590 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-44e39e94-925d-42dd-afd7-4e0ccb329284 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:17:42.893532 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1387/2789] 10.222.0.46 () {58 vars in 1232 bytes} [Thu May 26 15:17:42 2022] DELETE /identity/v3/projects/78466c91318f41baa08bff10235adb75 => generated 0 bytes in 446 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:17:42.981271 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba07f2e-f81a-4bb2-a3c2-9d1a4f2bf5df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:17:42.983246 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba07f2e-f81a-4bb2-a3c2-9d1a4f2bf5df admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:17:42.983768 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba07f2e-f81a-4bb2-a3c2-9d1a4f2bf5df admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:17:42.984288 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba07f2e-f81a-4bb2-a3c2-9d1a4f2bf5df admin admin] PATH_INFO: `/v3/projects/880800861cdf42b485f547696cf9ac4c/users/cb2581cad37e4ad9aff50c8f38e7c18d/roles/a2e551773cbd4df084020676865b8222` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:17:43.050878 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba07f2e-f81a-4bb2-a3c2-9d1a4f2bf5df admin admin] RBAC: Authorizing `identity:create_grant(project_id=880800861cdf42b485f547696cf9ac4c, user_id=cb2581cad37e4ad9aff50c8f38e7c18d, role_id=a2e551773cbd4df084020676865b8222)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:17:43.057077 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba07f2e-f81a-4bb2-a3c2-9d1a4f2bf5df admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:17:43.098340 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1403/2790] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 15:17:42 2022] PUT /identity/v3/projects/880800861cdf42b485f547696cf9ac4c/users/cb2581cad37e4ad9aff50c8f38e7c18d/roles/a2e551773cbd4df084020676865b8222 => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 15:17:43.104783 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e39e94-925d-42dd-afd7-4e0ccb329284 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:17:43.107539 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-44e39e94-925d-42dd-afd7-4e0ccb329284 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:17:43.164524 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e39e94-925d-42dd-afd7-4e0ccb329284 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:17:43.241734 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00f76e7-5c90-4195-994c-0f2e64775f94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:17:43.242658 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00f76e7-5c90-4195-994c-0f2e64775f94 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:17:43.242942 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00f76e7-5c90-4195-994c-0f2e64775f94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:17:43.243217 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00f76e7-5c90-4195-994c-0f2e64775f94 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:17:43.244010 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00f76e7-5c90-4195-994c-0f2e64775f94 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:17:43.245867 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00f76e7-5c90-4195-994c-0f2e64775f94 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:17:43.263620 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1388/2791] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 15:17:43 2022] GET /identity/v3/roles => generated 1420 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 15:17:43.265275 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba07f2e-f81a-4bb2-a3c2-9d1a4f2bf5df admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:17:43.279490 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba07f2e-f81a-4bb2-a3c2-9d1a4f2bf5df admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:17:43.286770 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85de7d44-e465-4c6b-88e7-02e7b584074d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:17:43.288085 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85de7d44-e465-4c6b-88e7-02e7b584074d admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:17:43.288556 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85de7d44-e465-4c6b-88e7-02e7b584074d admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:17:43.289023 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85de7d44-e465-4c6b-88e7-02e7b584074d admin admin] PATH_INFO: `/v3/projects/880800861cdf42b485f547696cf9ac4c/users/cb2581cad37e4ad9aff50c8f38e7c18d/roles/a2e551773cbd4df084020676865b8222` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:17:43.291634 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85de7d44-e465-4c6b-88e7-02e7b584074d admin admin] RBAC: Authorizing `identity:create_grant(project_id=880800861cdf42b485f547696cf9ac4c, user_id=cb2581cad37e4ad9aff50c8f38e7c18d, role_id=a2e551773cbd4df084020676865b8222)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:17:43.295595 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85de7d44-e465-4c6b-88e7-02e7b584074d admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:17:43.308956 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1404/2792] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 15:17:43 2022] PUT /identity/v3/projects/880800861cdf42b485f547696cf9ac4c/users/cb2581cad37e4ad9aff50c8f38e7c18d/roles/a2e551773cbd4df084020676865b8222 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 15:17:43.323343 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95aed479-2bf3-43f2-8712-6d852f3a1b12 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:17:43.323640 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95aed479-2bf3-43f2-8712-6d852f3a1b12 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:17:43.323924 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95aed479-2bf3-43f2-8712-6d852f3a1b12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:17:43.366131 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-95aed479-2bf3-43f2-8712-6d852f3a1b12 None None] MFA Rules not processed for user `cb2581cad37e4ad9aff50c8f38e7c18d`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:17:43.367427 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-95aed479-2bf3-43f2-8712-6d852f3a1b12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:17:43.556967 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1389/2793] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:17:43 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:17:43.586626 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85de7d44-e465-4c6b-88e7-02e7b584074d admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:17:43.588647 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-85de7d44-e465-4c6b-88e7-02e7b584074d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:17:43.748372 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85de7d44-e465-4c6b-88e7-02e7b584074d admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:17:43.765265 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0e1784-aaa3-4f60-89f0-b3c9afdaac25 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5b5b35614a547d783e6e77ddb8c1c03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:17:43.766120 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0e1784-aaa3-4f60-89f0-b3c9afdaac25 service cinder] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:17:43.766409 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0e1784-aaa3-4f60-89f0-b3c9afdaac25 service cinder] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:17:43.766683 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0e1784-aaa3-4f60-89f0-b3c9afdaac25 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:17:43.794197 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0e1784-aaa3-4f60-89f0-b3c9afdaac25 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:17:43.796492 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0e1784-aaa3-4f60-89f0-b3c9afdaac25 service cinder] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:17:43.809374 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:17:43.814581 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1405/2794] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:17:43 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 15:17:43.862355 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95aed479-2bf3-43f2-8712-6d852f3a1b12 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:17:43.976921 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95aed479-2bf3-43f2-8712-6d852f3a1b12 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:17:43.987707 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015e5377-9b1e-4cce-82e5-44f57c5e8963 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:17:43.990975 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015e5377-9b1e-4cce-82e5-44f57c5e8963 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:17:43.991612 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015e5377-9b1e-4cce-82e5-44f57c5e8963 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:17:43.992057 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015e5377-9b1e-4cce-82e5-44f57c5e8963 admin admin] PATH_INFO: `/v3/users/4a4e64641afd484f9fc4cf7fc8007ea2` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:17:43.994065 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015e5377-9b1e-4cce-82e5-44f57c5e8963 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4a4e64641afd484f9fc4cf7fc8007ea2)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:17:43.996949 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015e5377-9b1e-4cce-82e5-44f57c5e8963 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:17:44.016990 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=97,keystone:INSERT=1,keystone:DELETE=2 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:17:44.102981 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-015e5377-9b1e-4cce-82e5-44f57c5e8963 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4a4e64641afd484f9fc4cf7fc8007ea2'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:17:44.123494 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-015e5377-9b1e-4cce-82e5-44f57c5e8963 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4a4e64641afd484f9fc4cf7fc8007ea2'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:17:44.124823 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-015e5377-9b1e-4cce-82e5-44f57c5e8963 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4a4e64641afd484f9fc4cf7fc8007ea2'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:17:44.154822 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-015e5377-9b1e-4cce-82e5-44f57c5e8963 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4a4e64641afd484f9fc4cf7fc8007ea2'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:17:44.172609 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-015e5377-9b1e-4cce-82e5-44f57c5e8963 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4a4e64641afd484f9fc4cf7fc8007ea2'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:17:44.188213 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1390/2795] 10.222.0.46 () {58 vars in 1223 bytes} [Thu May 26 15:17:43 2022] DELETE /identity/v3/users/4a4e64641afd484f9fc4cf7fc8007ea2 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:17:44.188849 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dfec3c-bb4d-4194-a160-c636bf66f98e None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:17:44.189135 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dfec3c-bb4d-4194-a160-c636bf66f98e None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:17:44.189842 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5dfec3c-bb4d-4194-a160-c636bf66f98e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:17:44.241625 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-c5dfec3c-bb4d-4194-a160-c636bf66f98e None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:17:44.247748 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-c5dfec3c-bb4d-4194-a160-c636bf66f98e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:17:44.455079 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015e5377-9b1e-4cce-82e5-44f57c5e8963 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:17:44.457009 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-015e5377-9b1e-4cce-82e5-44f57c5e8963 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:17:44.462807 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1406/2796] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:17:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 275 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:17:44.485532 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015e5377-9b1e-4cce-82e5-44f57c5e8963 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:17:44.495164 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03636992-ae22-4d46-9a75-17744191000c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:17:44.495591 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03636992-ae22-4d46-9a75-17744191000c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:17:44.496799 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03636992-ae22-4d46-9a75-17744191000c admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:17:44.496799 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03636992-ae22-4d46-9a75-17744191000c admin admin] PATH_INFO: `/v3/users/e4afc097467f4c26a6d4da0e48b8303f` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:17:44.497807 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03636992-ae22-4d46-9a75-17744191000c admin admin] RBAC: Authorizing `identity:delete_user(user_id=e4afc097467f4c26a6d4da0e48b8303f)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:17:44.500571 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03636992-ae22-4d46-9a75-17744191000c admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:17:44.529590 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dfec3c-bb4d-4194-a160-c636bf66f98e None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:17:44.531504 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-c5dfec3c-bb4d-4194-a160-c636bf66f98e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:17:44.630351 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-03636992-ae22-4d46-9a75-17744191000c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e4afc097467f4c26a6d4da0e48b8303f'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:17:44.649607 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-03636992-ae22-4d46-9a75-17744191000c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e4afc097467f4c26a6d4da0e48b8303f'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:17:44.652963 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-03636992-ae22-4d46-9a75-17744191000c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e4afc097467f4c26a6d4da0e48b8303f'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:17:44.682160 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-03636992-ae22-4d46-9a75-17744191000c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e4afc097467f4c26a6d4da0e48b8303f'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:17:44.690675 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-03636992-ae22-4d46-9a75-17744191000c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e4afc097467f4c26a6d4da0e48b8303f'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:17:44.699713 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1391/2797] 10.222.0.46 () {58 vars in 1223 bytes} [Thu May 26 15:17:44 2022] DELETE /identity/v3/users/e4afc097467f4c26a6d4da0e48b8303f => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:17:44.700572 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5dfec3c-bb4d-4194-a160-c636bf66f98e None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:17:44.822774 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a57e2b7-be15-443d-8000-4c1ca30ebf64 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0156a2f6f3794f479e76b85bdbbe5c26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:17:44.825291 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a57e2b7-be15-443d-8000-4c1ca30ebf64 service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:17:44.826355 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a57e2b7-be15-443d-8000-4c1ca30ebf64 service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:17:44.827192 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a57e2b7-be15-443d-8000-4c1ca30ebf64 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:17:44.830611 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-1a57e2b7-be15-443d-8000-4c1ca30ebf64 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:17:44.972758 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a57e2b7-be15-443d-8000-4c1ca30ebf64 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:17:44.975040 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a57e2b7-be15-443d-8000-4c1ca30ebf64 service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:17:44.999026 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1407/2798] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:17:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 470 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 15:17:45.087436 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03636992-ae22-4d46-9a75-17744191000c admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:17:45.090286 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-03636992-ae22-4d46-9a75-17744191000c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:17:45.141204 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03636992-ae22-4d46-9a75-17744191000c admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:17:45.148868 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bb5ed3-6616-4057-81b6-568d770f2beb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:17:45.150411 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bb5ed3-6616-4057-81b6-568d770f2beb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:17:45.150907 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bb5ed3-6616-4057-81b6-568d770f2beb admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:17:45.151367 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bb5ed3-6616-4057-81b6-568d770f2beb admin admin] PATH_INFO: `/v3/projects/9c5cd1d25e6146bd93bd90286f668b9d` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:17:45.153791 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23bb5ed3-6616-4057-81b6-568d770f2beb admin admin] RBAC: Authorizing `identity:delete_project(project_id=9c5cd1d25e6146bd93bd90286f668b9d)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:17:45.159723 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23bb5ed3-6616-4057-81b6-568d770f2beb admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:17:45.300699 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-23bb5ed3-6616-4057-81b6-568d770f2beb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9c5cd1d25e6146bd93bd90286f668b9d', 'request_id': 'req-23bb5ed3-6616-4057-81b6-568d770f2beb'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:17:45.327452 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-23bb5ed3-6616-4057-81b6-568d770f2beb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9c5cd1d25e6146bd93bd90286f668b9d', 'request_id': 'req-23bb5ed3-6616-4057-81b6-568d770f2beb'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:17:45.335303 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=74,keystone:DELETE=24,keystone:INSERT=2 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:17:45.362907 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-23bb5ed3-6616-4057-81b6-568d770f2beb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9c5cd1d25e6146bd93bd90286f668b9d', 'request_id': 'req-23bb5ed3-6616-4057-81b6-568d770f2beb'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:17:45.410921 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-23bb5ed3-6616-4057-81b6-568d770f2beb admin admin] The token cache is being invalidate because project 9c5cd1d25e6146bd93bd90286f668b9d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 15:17:45.411296 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-23bb5ed3-6616-4057-81b6-568d770f2beb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:17:45.412653 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-23bb5ed3-6616-4057-81b6-568d770f2beb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:17:45.414983 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1392/2799] 10.222.0.46 () {58 vars in 1232 bytes} [Thu May 26 15:17:45 2022] DELETE /identity/v3/projects/9c5cd1d25e6146bd93bd90286f668b9d => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 15:17:45.421758 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a57e2b7-be15-443d-8000-4c1ca30ebf64 service neutron] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:17:45.451533 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a57e2b7-be15-443d-8000-4c1ca30ebf64 service neutron] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:17:45.465758 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=100 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:17:45.622501 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99044281-1e27-434e-8398-76aac06fa0d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:17:45.624645 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99044281-1e27-434e-8398-76aac06fa0d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:17:45.625169 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99044281-1e27-434e-8398-76aac06fa0d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:17:45.626179 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99044281-1e27-434e-8398-76aac06fa0d2 admin admin] PATH_INFO: `/v3/users/ea4c4d48b2174f24a937002d5eb19d77` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:17:45.628320 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99044281-1e27-434e-8398-76aac06fa0d2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ea4c4d48b2174f24a937002d5eb19d77)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:17:45.633133 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99044281-1e27-434e-8398-76aac06fa0d2 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:17:45.725813 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-99044281-1e27-434e-8398-76aac06fa0d2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ea4c4d48b2174f24a937002d5eb19d77'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:17:45.738177 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-99044281-1e27-434e-8398-76aac06fa0d2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ea4c4d48b2174f24a937002d5eb19d77'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:17:45.739497 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-99044281-1e27-434e-8398-76aac06fa0d2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ea4c4d48b2174f24a937002d5eb19d77'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:17:45.770162 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-99044281-1e27-434e-8398-76aac06fa0d2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ea4c4d48b2174f24a937002d5eb19d77'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:17:45.787384 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-99044281-1e27-434e-8398-76aac06fa0d2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ea4c4d48b2174f24a937002d5eb19d77'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:17:45.797504 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1408/2800] 10.222.0.46 () {58 vars in 1223 bytes} [Thu May 26 15:17:45 2022] DELETE /identity/v3/users/ea4c4d48b2174f24a937002d5eb19d77 => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:17:45.902989 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bb5ed3-6616-4057-81b6-568d770f2beb admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:17:45.904826 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-23bb5ed3-6616-4057-81b6-568d770f2beb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:17:45.926327 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23bb5ed3-6616-4057-81b6-568d770f2beb admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:17:46.062091 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99044281-1e27-434e-8398-76aac06fa0d2 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:17:46.063708 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-99044281-1e27-434e-8398-76aac06fa0d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:17:46.095415 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bfbe19-1f94-41d5-8802-635a62a95810 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:17:46.096886 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bfbe19-1f94-41d5-8802-635a62a95810 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:17:46.096886 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bfbe19-1f94-41d5-8802-635a62a95810 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:17:46.097157 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05bfbe19-1f94-41d5-8802-635a62a95810 admin admin] PATH_INFO: `/v3/projects/72d3efe0d5d646bbafdaad6598bbacb7` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:17:46.098545 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bfbe19-1f94-41d5-8802-635a62a95810 admin admin] RBAC: Authorizing `identity:delete_project(project_id=72d3efe0d5d646bbafdaad6598bbacb7)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:17:46.102015 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05bfbe19-1f94-41d5-8802-635a62a95810 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:17:46.197279 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99044281-1e27-434e-8398-76aac06fa0d2 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:17:46.206694 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-05bfbe19-1f94-41d5-8802-635a62a95810 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '72d3efe0d5d646bbafdaad6598bbacb7', 'request_id': 'req-05bfbe19-1f94-41d5-8802-635a62a95810'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:17:46.234087 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-05bfbe19-1f94-41d5-8802-635a62a95810 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '72d3efe0d5d646bbafdaad6598bbacb7', 'request_id': 'req-05bfbe19-1f94-41d5-8802-635a62a95810'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:17:46.249379 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-05bfbe19-1f94-41d5-8802-635a62a95810 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '72d3efe0d5d646bbafdaad6598bbacb7', 'request_id': 'req-05bfbe19-1f94-41d5-8802-635a62a95810'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:17:46.266642 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:17:46.285178 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-05bfbe19-1f94-41d5-8802-635a62a95810 admin admin] The token cache is being invalidate because project 72d3efe0d5d646bbafdaad6598bbacb7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 15:17:46.285873 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-05bfbe19-1f94-41d5-8802-635a62a95810 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:17:46.288583 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-05bfbe19-1f94-41d5-8802-635a62a95810 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:17:46.293552 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1393/2801] 10.222.0.46 () {58 vars in 1232 bytes} [Thu May 26 15:17:45 2022] DELETE /identity/v3/projects/72d3efe0d5d646bbafdaad6598bbacb7 => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:17:46.297154 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bfbe19-1f94-41d5-8802-635a62a95810 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:17:46.304078 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-05bfbe19-1f94-41d5-8802-635a62a95810 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:17:46.320325 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3057d38e-b7fa-46e7-ab01-bb31f2a4a0e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:17:46.320325 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3057d38e-b7fa-46e7-ab01-bb31f2a4a0e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:17:46.320804 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3057d38e-b7fa-46e7-ab01-bb31f2a4a0e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:17:46.320952 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3057d38e-b7fa-46e7-ab01-bb31f2a4a0e9 admin admin] PATH_INFO: `/v3/projects/6307f64212a64a84b5369682457e94db` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:17:46.325224 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3057d38e-b7fa-46e7-ab01-bb31f2a4a0e9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6307f64212a64a84b5369682457e94db)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:17:46.327287 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3057d38e-b7fa-46e7-ab01-bb31f2a4a0e9 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:17:46.492981 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05bfbe19-1f94-41d5-8802-635a62a95810 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:17:46.501133 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-3057d38e-b7fa-46e7-ab01-bb31f2a4a0e9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6307f64212a64a84b5369682457e94db', 'request_id': 'req-3057d38e-b7fa-46e7-ab01-bb31f2a4a0e9'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:17:46.517242 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-3057d38e-b7fa-46e7-ab01-bb31f2a4a0e9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6307f64212a64a84b5369682457e94db', 'request_id': 'req-3057d38e-b7fa-46e7-ab01-bb31f2a4a0e9'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:17:46.528260 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:17:46.529921 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-3057d38e-b7fa-46e7-ab01-bb31f2a4a0e9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6307f64212a64a84b5369682457e94db', 'request_id': 'req-3057d38e-b7fa-46e7-ab01-bb31f2a4a0e9'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:17:46.576354 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-3057d38e-b7fa-46e7-ab01-bb31f2a4a0e9 admin admin] The token cache is being invalidate because project 6307f64212a64a84b5369682457e94db was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 15:17:46.576818 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-3057d38e-b7fa-46e7-ab01-bb31f2a4a0e9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:17:46.578029 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-3057d38e-b7fa-46e7-ab01-bb31f2a4a0e9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:17:46.581509 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1409/2802] 10.222.0.46 () {58 vars in 1232 bytes} [Thu May 26 15:17:46 2022] DELETE /identity/v3/projects/6307f64212a64a84b5369682457e94db => generated 0 bytes in 518 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 15:17:46.631308 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf803afe-a49e-4c78-af4f-d2c8509d4f29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:17:46.633195 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf803afe-a49e-4c78-af4f-d2c8509d4f29 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:17:46.633778 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf803afe-a49e-4c78-af4f-d2c8509d4f29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:17:46.634226 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf803afe-a49e-4c78-af4f-d2c8509d4f29 admin admin] PATH_INFO: `/v3/users/cb2581cad37e4ad9aff50c8f38e7c18d` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:17:46.636113 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf803afe-a49e-4c78-af4f-d2c8509d4f29 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cb2581cad37e4ad9aff50c8f38e7c18d)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:17:46.637815 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf803afe-a49e-4c78-af4f-d2c8509d4f29 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:17:46.676500 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91632322-6b41-4407-a3ba-ebac7c92d36d None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:17:46.676807 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91632322-6b41-4407-a3ba-ebac7c92d36d None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:17:46.677061 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91632322-6b41-4407-a3ba-ebac7c92d36d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:17:46.717330 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-91632322-6b41-4407-a3ba-ebac7c92d36d None None] MFA Rules not processed for user `2991d18c077c470eab993644b445bce9`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:17:46.718837 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-91632322-6b41-4407-a3ba-ebac7c92d36d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:17:46.734792 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-cf803afe-a49e-4c78-af4f-d2c8509d4f29 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cb2581cad37e4ad9aff50c8f38e7c18d'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:17:46.750070 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-cf803afe-a49e-4c78-af4f-d2c8509d4f29 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cb2581cad37e4ad9aff50c8f38e7c18d'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:17:46.751674 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-cf803afe-a49e-4c78-af4f-d2c8509d4f29 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cb2581cad37e4ad9aff50c8f38e7c18d'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:17:46.787649 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-cf803afe-a49e-4c78-af4f-d2c8509d4f29 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cb2581cad37e4ad9aff50c8f38e7c18d'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:17:46.802039 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-cf803afe-a49e-4c78-af4f-d2c8509d4f29 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cb2581cad37e4ad9aff50c8f38e7c18d'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:17:46.837325 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1394/2803] 10.222.0.46 () {58 vars in 1223 bytes} [Thu May 26 15:17:46 2022] DELETE /identity/v3/users/cb2581cad37e4ad9aff50c8f38e7c18d => generated 0 bytes in 518 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 15:17:46.847503 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1410/2804] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:17:46 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:17:46.865700 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf803afe-a49e-4c78-af4f-d2c8509d4f29 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:17:46.867442 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-cf803afe-a49e-4c78-af4f-d2c8509d4f29 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:17:46.966004 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf803afe-a49e-4c78-af4f-d2c8509d4f29 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:17:46.973579 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654f5dac-173f-4908-8f93-370ce2a0841c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6639353c70449d1b3d10cd6487c3b00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:17:46.974287 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654f5dac-173f-4908-8f93-370ce2a0841c service nova] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:17:46.974628 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654f5dac-173f-4908-8f93-370ce2a0841c service nova] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:17:46.974900 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654f5dac-173f-4908-8f93-370ce2a0841c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:17:46.976827 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-654f5dac-173f-4908-8f93-370ce2a0841c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:17:47.070139 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:17:47.095426 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654f5dac-173f-4908-8f93-370ce2a0841c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:17:47.097969 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654f5dac-173f-4908-8f93-370ce2a0841c service nova] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:17:47.115073 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1395/2805] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:17:46 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 250 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 15:17:47.232520 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91632322-6b41-4407-a3ba-ebac7c92d36d None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:17:47.234056 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-91632322-6b41-4407-a3ba-ebac7c92d36d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:17:47.262806 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91632322-6b41-4407-a3ba-ebac7c92d36d None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:17:47.330368 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:17:47.350524 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a6514d-4f23-4793-a37d-b64cc4e1339a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:17:47.351411 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a6514d-4f23-4793-a37d-b64cc4e1339a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:17:47.352823 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a6514d-4f23-4793-a37d-b64cc4e1339a admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:17:47.352823 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a6514d-4f23-4793-a37d-b64cc4e1339a admin admin] PATH_INFO: `/v3/projects/880800861cdf42b485f547696cf9ac4c` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:17:47.353860 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a6514d-4f23-4793-a37d-b64cc4e1339a admin admin] RBAC: Authorizing `identity:delete_project(project_id=880800861cdf42b485f547696cf9ac4c)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:17:47.356486 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a6514d-4f23-4793-a37d-b64cc4e1339a admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:17:47.426793 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-b3a6514d-4f23-4793-a37d-b64cc4e1339a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '880800861cdf42b485f547696cf9ac4c', 'request_id': 'req-b3a6514d-4f23-4793-a37d-b64cc4e1339a'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:17:47.436342 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-b3a6514d-4f23-4793-a37d-b64cc4e1339a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '880800861cdf42b485f547696cf9ac4c', 'request_id': 'req-b3a6514d-4f23-4793-a37d-b64cc4e1339a'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:17:47.444485 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-b3a6514d-4f23-4793-a37d-b64cc4e1339a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '880800861cdf42b485f547696cf9ac4c', 'request_id': 'req-b3a6514d-4f23-4793-a37d-b64cc4e1339a'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:17:47.459262 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-b3a6514d-4f23-4793-a37d-b64cc4e1339a admin admin] The token cache is being invalidate because project 880800861cdf42b485f547696cf9ac4c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 15:17:47.459548 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-b3a6514d-4f23-4793-a37d-b64cc4e1339a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:17:47.460225 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-b3a6514d-4f23-4793-a37d-b64cc4e1339a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:17:47.462453 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1411/2806] 10.222.0.46 () {58 vars in 1232 bytes} [Thu May 26 15:17:47 2022] DELETE /identity/v3/projects/880800861cdf42b485f547696cf9ac4c => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:17:47.467429 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efaea18d-276c-4f28-8955-58e8a3dc57d2 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:17:47.468188 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efaea18d-276c-4f28-8955-58e8a3dc57d2 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:17:47.468188 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efaea18d-276c-4f28-8955-58e8a3dc57d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:17:47.488651 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-efaea18d-276c-4f28-8955-58e8a3dc57d2 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:17:47.490728 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-efaea18d-276c-4f28-8955-58e8a3dc57d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:17:47.589278 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1396/2807] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:17:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:17:47.591933 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb2afcf-a709-4bc6-9691-fd80cd75eb56 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:17:47.592301 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb2afcf-a709-4bc6-9691-fd80cd75eb56 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:17:47.592792 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb2afcf-a709-4bc6-9691-fd80cd75eb56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:17:47.616765 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-3eb2afcf-a709-4bc6-9691-fd80cd75eb56 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:17:47.618498 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-3eb2afcf-a709-4bc6-9691-fd80cd75eb56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:17:47.644508 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1412/2808] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:17:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:17:47.649467 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efaea18d-276c-4f28-8955-58e8a3dc57d2 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:17:47.665170 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efaea18d-276c-4f28-8955-58e8a3dc57d2 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:17:47.670781 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464bd3b9-4137-4fad-b5be-012f1dd62b41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:17:47.671826 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464bd3b9-4137-4fad-b5be-012f1dd62b41 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:17:47.672063 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464bd3b9-4137-4fad-b5be-012f1dd62b41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:17:47.672342 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464bd3b9-4137-4fad-b5be-012f1dd62b41 admin admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:17:47.673094 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464bd3b9-4137-4fad-b5be-012f1dd62b41 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 15:17:47.673912 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464bd3b9-4137-4fad-b5be-012f1dd62b41 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:17:47.675379 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464bd3b9-4137-4fad-b5be-012f1dd62b41 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:17:47.704586 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1397/2809] 10.222.0.46 () {58 vars in 1152 bytes} [Thu May 26 15:17:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 15:17:47.706725 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb2afcf-a709-4bc6-9691-fd80cd75eb56 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:17:47.717170 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb2afcf-a709-4bc6-9691-fd80cd75eb56 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:17:47.722209 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd47133-ac3d-4399-a543-fc9520815918 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:17:47.723060 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd47133-ac3d-4399-a543-fc9520815918 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:17:47.723397 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd47133-ac3d-4399-a543-fc9520815918 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:17:47.723658 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd47133-ac3d-4399-a543-fc9520815918 admin admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:17:47.725235 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd47133-ac3d-4399-a543-fc9520815918 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:17:47.727902 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd47133-ac3d-4399-a543-fc9520815918 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:17:47.764920 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1413/2810] 10.222.0.46 () {60 vars in 1152 bytes} [Thu May 26 15:17:47 2022] POST /identity/v3/projects => generated 387 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 15:17:47.769511 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464bd3b9-4137-4fad-b5be-012f1dd62b41 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:17:47.789727 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464bd3b9-4137-4fad-b5be-012f1dd62b41 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:17:47.874768 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef80268-d0ff-42ee-b8f0-83104baaf830 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:17:47.875436 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef80268-d0ff-42ee-b8f0-83104baaf830 admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:17:47.875945 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef80268-d0ff-42ee-b8f0-83104baaf830 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:17:47.876449 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef80268-d0ff-42ee-b8f0-83104baaf830 admin admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:17:47.877332 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef80268-d0ff-42ee-b8f0-83104baaf830 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:17:47.879762 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef80268-d0ff-42ee-b8f0-83104baaf830 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:17:47.924856 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1398/2811] 10.222.0.46 () {60 vars in 1143 bytes} [Thu May 26 15:17:47 2022] POST /identity/v3/users => generated 294 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 15:17:47.925632 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd47133-ac3d-4399-a543-fc9520815918 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:17:47.939585 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd47133-ac3d-4399-a543-fc9520815918 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:17:47.945933 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2587293c-3867-4fdf-989d-94d8390ee639 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:17:47.946813 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2587293c-3867-4fdf-989d-94d8390ee639 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:17:47.947280 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2587293c-3867-4fdf-989d-94d8390ee639 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:17:47.947710 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2587293c-3867-4fdf-989d-94d8390ee639 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:17:47.948610 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2587293c-3867-4fdf-989d-94d8390ee639 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:17:47.950456 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2587293c-3867-4fdf-989d-94d8390ee639 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:17:47.965651 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1414/2812] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 15:17:47 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 15:17:47.970124 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef80268-d0ff-42ee-b8f0-83104baaf830 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:17:47.984234 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef80268-d0ff-42ee-b8f0-83104baaf830 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:17:47.995322 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a873d67c-02a0-43a4-838d-7d9ba766ab3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:17:47.996029 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a873d67c-02a0-43a4-838d-7d9ba766ab3b admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:17:47.996320 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a873d67c-02a0-43a4-838d-7d9ba766ab3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:17:47.996605 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a873d67c-02a0-43a4-838d-7d9ba766ab3b admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:17:47.997700 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a873d67c-02a0-43a4-838d-7d9ba766ab3b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:17:48.000608 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a873d67c-02a0-43a4-838d-7d9ba766ab3b admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:17:48.011863 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1399/2813] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 15:17:47 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 15:17:48.016210 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2587293c-3867-4fdf-989d-94d8390ee639 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:17:48.035691 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2587293c-3867-4fdf-989d-94d8390ee639 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:17:48.043462 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10783c87-75dd-4613-87e5-08c71a909293 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:17:48.045206 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10783c87-75dd-4613-87e5-08c71a909293 admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:17:48.045685 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10783c87-75dd-4613-87e5-08c71a909293 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:17:48.046280 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10783c87-75dd-4613-87e5-08c71a909293 admin admin] PATH_INFO: `/v3/projects/575e35684b064ccfaca662ea8ee3014d/users/f84d88b913634ca1a02cf1b006b98110/roles/d4088d1845a24fbd9bd8975ce0b1d3d0` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:17:48.071904 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10783c87-75dd-4613-87e5-08c71a909293 admin admin] RBAC: Authorizing `identity:create_grant(project_id=575e35684b064ccfaca662ea8ee3014d, user_id=f84d88b913634ca1a02cf1b006b98110, role_id=d4088d1845a24fbd9bd8975ce0b1d3d0)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:17:48.075482 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10783c87-75dd-4613-87e5-08c71a909293 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:17:48.097687 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1415/2814] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 15:17:48 2022] PUT /identity/v3/projects/575e35684b064ccfaca662ea8ee3014d/users/f84d88b913634ca1a02cf1b006b98110/roles/d4088d1845a24fbd9bd8975ce0b1d3d0 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:17:48.110677 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9390042-d627-4934-80c6-a74e202e9117 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:17:48.111141 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9390042-d627-4934-80c6-a74e202e9117 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:17:48.111141 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9390042-d627-4934-80c6-a74e202e9117 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:17:48.143616 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-d9390042-d627-4934-80c6-a74e202e9117 None None] MFA Rules not processed for user `f84d88b913634ca1a02cf1b006b98110`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:17:48.144875 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-d9390042-d627-4934-80c6-a74e202e9117 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:17:48.196057 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:17:48.244989 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1400/2815] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:17:48 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:17:48.252099 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10783c87-75dd-4613-87e5-08c71a909293 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:17:48.362149 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10783c87-75dd-4613-87e5-08c71a909293 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:17:48.373667 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e35d87b-c6e0-4437-b288-3ce0368a66cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:17:48.375865 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e35d87b-c6e0-4437-b288-3ce0368a66cf admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:17:48.375865 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e35d87b-c6e0-4437-b288-3ce0368a66cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:17:48.378576 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e35d87b-c6e0-4437-b288-3ce0368a66cf admin admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:17:48.378576 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e35d87b-c6e0-4437-b288-3ce0368a66cf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:17:48.380113 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e35d87b-c6e0-4437-b288-3ce0368a66cf admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:17:48.406605 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:17:48.416018 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1416/2816] 10.222.0.46 () {60 vars in 1152 bytes} [Thu May 26 15:17:48 2022] POST /identity/v3/projects => generated 385 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 15:17:48.421375 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9390042-d627-4934-80c6-a74e202e9117 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:17:48.450566 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9390042-d627-4934-80c6-a74e202e9117 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:17:48.544103 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7689a549-5fb7-42f9-99cb-c3cf8034bda7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:17:48.544900 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7689a549-5fb7-42f9-99cb-c3cf8034bda7 admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:17:48.545814 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7689a549-5fb7-42f9-99cb-c3cf8034bda7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:17:48.546140 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7689a549-5fb7-42f9-99cb-c3cf8034bda7 admin admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:17:48.547247 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7689a549-5fb7-42f9-99cb-c3cf8034bda7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:17:48.549575 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7689a549-5fb7-42f9-99cb-c3cf8034bda7 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:17:48.576035 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1401/2817] 10.222.0.46 () {60 vars in 1143 bytes} [Thu May 26 15:17:48 2022] POST /identity/v3/users => generated 293 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 26 15:17:48.580583 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e35d87b-c6e0-4437-b288-3ce0368a66cf admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:17:48.594123 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e35d87b-c6e0-4437-b288-3ce0368a66cf admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:17:48.600014 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5bf088-8364-4a9f-b108-e4ffb78ba883 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:17:48.600895 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5bf088-8364-4a9f-b108-e4ffb78ba883 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:17:48.601314 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5bf088-8364-4a9f-b108-e4ffb78ba883 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:17:48.601720 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5bf088-8364-4a9f-b108-e4ffb78ba883 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:17:48.603279 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5bf088-8364-4a9f-b108-e4ffb78ba883 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:17:48.605110 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5bf088-8364-4a9f-b108-e4ffb78ba883 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:17:48.618906 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1417/2818] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 15:17:48 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 15:17:48.623260 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7689a549-5fb7-42f9-99cb-c3cf8034bda7 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:17:48.636079 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7689a549-5fb7-42f9-99cb-c3cf8034bda7 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:17:48.641152 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad38bb8-2866-4e5c-9502-208d025e36f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:17:48.641998 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad38bb8-2866-4e5c-9502-208d025e36f1 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:17:48.642311 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad38bb8-2866-4e5c-9502-208d025e36f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:17:48.642589 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad38bb8-2866-4e5c-9502-208d025e36f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:17:48.643341 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad38bb8-2866-4e5c-9502-208d025e36f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:17:48.644928 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad38bb8-2866-4e5c-9502-208d025e36f1 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:17:48.659216 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1402/2819] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 15:17:48 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 15:17:48.663775 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5bf088-8364-4a9f-b108-e4ffb78ba883 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:17:48.674001 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5bf088-8364-4a9f-b108-e4ffb78ba883 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:17:48.685623 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e264469-e601-431c-8f17-d2cd923b9176 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:17:48.686533 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e264469-e601-431c-8f17-d2cd923b9176 admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:17:48.686815 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e264469-e601-431c-8f17-d2cd923b9176 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:17:48.687093 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e264469-e601-431c-8f17-d2cd923b9176 admin admin] PATH_INFO: `/v3/projects/5ebc044d2ba246bd83b2205f7cb7dff6/users/9a83869cdbb44e14a81d234047d2ba3f/roles/d4088d1845a24fbd9bd8975ce0b1d3d0` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:17:48.703874 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e264469-e601-431c-8f17-d2cd923b9176 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ebc044d2ba246bd83b2205f7cb7dff6, user_id=9a83869cdbb44e14a81d234047d2ba3f, role_id=d4088d1845a24fbd9bd8975ce0b1d3d0)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:17:48.706621 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e264469-e601-431c-8f17-d2cd923b9176 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:17:48.722237 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1418/2820] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 15:17:48 2022] PUT /identity/v3/projects/5ebc044d2ba246bd83b2205f7cb7dff6/users/9a83869cdbb44e14a81d234047d2ba3f/roles/d4088d1845a24fbd9bd8975ce0b1d3d0 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:17:48.733871 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477624ee-7934-47da-a197-5ab124492824 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:17:48.734168 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477624ee-7934-47da-a197-5ab124492824 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:17:48.734455 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477624ee-7934-47da-a197-5ab124492824 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:17:48.765950 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-477624ee-7934-47da-a197-5ab124492824 None None] MFA Rules not processed for user `9a83869cdbb44e14a81d234047d2ba3f`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:17:48.767190 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-477624ee-7934-47da-a197-5ab124492824 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:17:48.857086 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1403/2821] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:17:48 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:17:48.862113 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e264469-e601-431c-8f17-d2cd923b9176 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:17:48.935576 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e264469-e601-431c-8f17-d2cd923b9176 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:17:48.957704 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deedda34-85b2-457e-8be0-578edcf6da48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:17:48.958276 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deedda34-85b2-457e-8be0-578edcf6da48 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:17:48.958716 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deedda34-85b2-457e-8be0-578edcf6da48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:17:48.959105 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deedda34-85b2-457e-8be0-578edcf6da48 admin admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:17:48.959948 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deedda34-85b2-457e-8be0-578edcf6da48 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:17:48.962419 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deedda34-85b2-457e-8be0-578edcf6da48 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:17:49.005099 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1419/2822] 10.222.0.46 () {60 vars in 1152 bytes} [Thu May 26 15:17:48 2022] POST /identity/v3/projects => generated 385 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 15:17:49.008473 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477624ee-7934-47da-a197-5ab124492824 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:17:49.021391 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477624ee-7934-47da-a197-5ab124492824 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:17:49.071955 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:17:49.089934 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c175996c-a68c-452d-8d6c-018884fe5617 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:17:49.090726 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c175996c-a68c-452d-8d6c-018884fe5617 admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:17:49.091086 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c175996c-a68c-452d-8d6c-018884fe5617 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:17:49.091441 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c175996c-a68c-452d-8d6c-018884fe5617 admin admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:17:49.092217 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c175996c-a68c-452d-8d6c-018884fe5617 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:17:49.094265 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c175996c-a68c-452d-8d6c-018884fe5617 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:17:49.118486 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1404/2823] 10.222.0.46 () {60 vars in 1143 bytes} [Thu May 26 15:17:49 2022] POST /identity/v3/users => generated 299 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 15:17:49.122923 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deedda34-85b2-457e-8be0-578edcf6da48 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:17:49.134784 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deedda34-85b2-457e-8be0-578edcf6da48 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:17:49.139555 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4de5b43-c36b-41db-82a0-b9420e2eb10f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:17:49.140231 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4de5b43-c36b-41db-82a0-b9420e2eb10f admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:17:49.140521 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4de5b43-c36b-41db-82a0-b9420e2eb10f admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:17:49.140792 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4de5b43-c36b-41db-82a0-b9420e2eb10f admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:17:49.141564 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4de5b43-c36b-41db-82a0-b9420e2eb10f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:17:49.143613 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4de5b43-c36b-41db-82a0-b9420e2eb10f admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:17:49.155313 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1420/2824] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 15:17:49 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 26 15:17:49.159863 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c175996c-a68c-452d-8d6c-018884fe5617 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:17:49.171810 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c175996c-a68c-452d-8d6c-018884fe5617 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:17:49.178377 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce1dbb0-c6c7-4501-b883-33509ba72ea5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:17:49.179735 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce1dbb0-c6c7-4501-b883-33509ba72ea5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:17:49.180021 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce1dbb0-c6c7-4501-b883-33509ba72ea5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:17:49.180308 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce1dbb0-c6c7-4501-b883-33509ba72ea5 admin admin] PATH_INFO: `/v3/projects/aa3b0d5f0a204f8598657c33374796fe/users/a633b3f1025b4f1a9fe0339f1ee12a39/roles/a2e551773cbd4df084020676865b8222` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:17:49.209288 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce1dbb0-c6c7-4501-b883-33509ba72ea5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa3b0d5f0a204f8598657c33374796fe, user_id=a633b3f1025b4f1a9fe0339f1ee12a39, role_id=a2e551773cbd4df084020676865b8222)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:17:49.212429 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce1dbb0-c6c7-4501-b883-33509ba72ea5 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:17:49.233100 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1405/2825] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 15:17:49 2022] PUT /identity/v3/projects/aa3b0d5f0a204f8598657c33374796fe/users/a633b3f1025b4f1a9fe0339f1ee12a39/roles/a2e551773cbd4df084020676865b8222 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:17:49.237236 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4de5b43-c36b-41db-82a0-b9420e2eb10f admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:17:49.249861 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4de5b43-c36b-41db-82a0-b9420e2eb10f admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:17:49.323610 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9349e55c-c8e4-4d01-a37c-9fd270da9355 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:17:49.324424 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9349e55c-c8e4-4d01-a37c-9fd270da9355 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:17:49.324581 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9349e55c-c8e4-4d01-a37c-9fd270da9355 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:17:49.324859 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9349e55c-c8e4-4d01-a37c-9fd270da9355 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:17:49.326207 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9349e55c-c8e4-4d01-a37c-9fd270da9355 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:17:49.327986 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9349e55c-c8e4-4d01-a37c-9fd270da9355 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:17:49.339389 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1421/2826] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 15:17:49 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 15:17:49.343612 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce1dbb0-c6c7-4501-b883-33509ba72ea5 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:17:49.357989 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce1dbb0-c6c7-4501-b883-33509ba72ea5 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:17:49.362748 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585c7fe4-3d21-49dd-aae1-c6bdf5c3305f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:17:49.363657 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585c7fe4-3d21-49dd-aae1-c6bdf5c3305f admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:17:49.363947 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585c7fe4-3d21-49dd-aae1-c6bdf5c3305f admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:17:49.364264 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-585c7fe4-3d21-49dd-aae1-c6bdf5c3305f admin admin] PATH_INFO: `/v3/projects/aa3b0d5f0a204f8598657c33374796fe/users/a633b3f1025b4f1a9fe0339f1ee12a39/roles/a2e551773cbd4df084020676865b8222` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:17:49.367125 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585c7fe4-3d21-49dd-aae1-c6bdf5c3305f admin admin] RBAC: Authorizing `identity:create_grant(project_id=aa3b0d5f0a204f8598657c33374796fe, user_id=a633b3f1025b4f1a9fe0339f1ee12a39, role_id=a2e551773cbd4df084020676865b8222)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:17:49.369815 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-585c7fe4-3d21-49dd-aae1-c6bdf5c3305f admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:17:49.383911 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1406/2827] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 15:17:49 2022] PUT /identity/v3/projects/aa3b0d5f0a204f8598657c33374796fe/users/a633b3f1025b4f1a9fe0339f1ee12a39/roles/a2e551773cbd4df084020676865b8222 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 15:17:49.463283 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846277da-bb5a-4879-b510-091e9caeb102 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:17:49.464048 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846277da-bb5a-4879-b510-091e9caeb102 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:17:49.464400 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846277da-bb5a-4879-b510-091e9caeb102 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:17:49.495137 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-846277da-bb5a-4879-b510-091e9caeb102 None None] MFA Rules not processed for user `a633b3f1025b4f1a9fe0339f1ee12a39`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:17:49.496517 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-846277da-bb5a-4879-b510-091e9caeb102 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:17:49.532885 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:17:49.591260 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1422/2828] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:17:49 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:17:49.604667 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585c7fe4-3d21-49dd-aae1-c6bdf5c3305f admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:17:49.606575 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-585c7fe4-3d21-49dd-aae1-c6bdf5c3305f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:17:49.681150 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-585c7fe4-3d21-49dd-aae1-c6bdf5c3305f admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:17:49.685095 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d769fce6-be26-4056-90e6-2cfd30bc5f23 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5b5b35614a547d783e6e77ddb8c1c03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:17:49.685783 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d769fce6-be26-4056-90e6-2cfd30bc5f23 service cinder] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:17:49.686067 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d769fce6-be26-4056-90e6-2cfd30bc5f23 service cinder] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:17:49.686342 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d769fce6-be26-4056-90e6-2cfd30bc5f23 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:17:49.696393 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d769fce6-be26-4056-90e6-2cfd30bc5f23 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:17:49.699270 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d769fce6-be26-4056-90e6-2cfd30bc5f23 service cinder] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:17:49.722839 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1407/2829] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:17:49 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 15:17:50.393825 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846277da-bb5a-4879-b510-091e9caeb102 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:17:50.395266 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-846277da-bb5a-4879-b510-091e9caeb102 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:17:50.409873 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846277da-bb5a-4879-b510-091e9caeb102 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:17:50.414295 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e369a73-e3f0-42ec-aa1c-7a94e9e56b09 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5b5b35614a547d783e6e77ddb8c1c03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:17:50.414937 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e369a73-e3f0-42ec-aa1c-7a94e9e56b09 service cinder] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:17:50.415221 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e369a73-e3f0-42ec-aa1c-7a94e9e56b09 service cinder] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:17:50.415501 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e369a73-e3f0-42ec-aa1c-7a94e9e56b09 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:17:50.509168 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e369a73-e3f0-42ec-aa1c-7a94e9e56b09 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:17:50.510867 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e369a73-e3f0-42ec-aa1c-7a94e9e56b09 service cinder] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:17:50.526530 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1423/2830] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:17:50 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 15:17:54.801302 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d769fce6-be26-4056-90e6-2cfd30bc5f23 service cinder] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:17:54.810978 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d769fce6-be26-4056-90e6-2cfd30bc5f23 service cinder] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:17:54.815450 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ad22c1-0259-4efe-952c-7a589340d952 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5b5b35614a547d783e6e77ddb8c1c03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:17:54.816173 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ad22c1-0259-4efe-952c-7a589340d952 service cinder] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:17:54.816460 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ad22c1-0259-4efe-952c-7a589340d952 service cinder] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:17:54.816834 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ad22c1-0259-4efe-952c-7a589340d952 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:17:54.892536 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ad22c1-0259-4efe-952c-7a589340d952 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:17:54.894055 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ad22c1-0259-4efe-952c-7a589340d952 service cinder] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:17:54.904313 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1408/2831] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:17:54 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 15:17:57.249798 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e369a73-e3f0-42ec-aa1c-7a94e9e56b09 service cinder] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:17:57.322264 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e369a73-e3f0-42ec-aa1c-7a94e9e56b09 service cinder] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:17:57.324864 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82ffbdd-c20f-4774-a593-b60deb5878ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:17:57.325859 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82ffbdd-c20f-4774-a593-b60deb5878ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:17:57.326246 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82ffbdd-c20f-4774-a593-b60deb5878ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:17:57.326641 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82ffbdd-c20f-4774-a593-b60deb5878ad admin admin] PATH_INFO: `/v3/users/f84d88b913634ca1a02cf1b006b98110` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:17:57.327968 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82ffbdd-c20f-4774-a593-b60deb5878ad admin admin] RBAC: Authorizing `identity:delete_user(user_id=f84d88b913634ca1a02cf1b006b98110)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:17:57.330082 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82ffbdd-c20f-4774-a593-b60deb5878ad admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:17:57.394195 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-a82ffbdd-c20f-4774-a593-b60deb5878ad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f84d88b913634ca1a02cf1b006b98110'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:17:57.406496 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-a82ffbdd-c20f-4774-a593-b60deb5878ad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f84d88b913634ca1a02cf1b006b98110'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:17:57.407165 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-a82ffbdd-c20f-4774-a593-b60deb5878ad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f84d88b913634ca1a02cf1b006b98110'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:17:57.413599 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:17:57.423026 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-a82ffbdd-c20f-4774-a593-b60deb5878ad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f84d88b913634ca1a02cf1b006b98110'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:17:57.438054 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-a82ffbdd-c20f-4774-a593-b60deb5878ad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f84d88b913634ca1a02cf1b006b98110'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:17:57.445410 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1424/2832] 10.222.0.46 () {58 vars in 1223 bytes} [Thu May 26 15:17:57 2022] DELETE /identity/v3/users/f84d88b913634ca1a02cf1b006b98110 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:17:57.450444 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264452fe-3f35-48e4-b304-5bc076a97ba1 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:17:57.451254 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264452fe-3f35-48e4-b304-5bc076a97ba1 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:17:57.451254 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264452fe-3f35-48e4-b304-5bc076a97ba1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:17:57.474356 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-264452fe-3f35-48e4-b304-5bc076a97ba1 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:17:57.477401 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-264452fe-3f35-48e4-b304-5bc076a97ba1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:17:57.513981 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:17:57.548216 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1409/2833] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:17:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:17:57.559738 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82ffbdd-c20f-4774-a593-b60deb5878ad admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:17:57.561858 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-a82ffbdd-c20f-4774-a593-b60deb5878ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:17:57.631844 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82ffbdd-c20f-4774-a593-b60deb5878ad admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:17:57.635584 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0762d6c-7376-45d1-94fe-3d561ef21da3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0156a2f6f3794f479e76b85bdbbe5c26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:17:57.638212 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0762d6c-7376-45d1-94fe-3d561ef21da3 service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:17:57.638212 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0762d6c-7376-45d1-94fe-3d561ef21da3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:17:57.638212 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0762d6c-7376-45d1-94fe-3d561ef21da3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:17:57.650188 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0762d6c-7376-45d1-94fe-3d561ef21da3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:17:57.652575 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0762d6c-7376-45d1-94fe-3d561ef21da3 service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:17:57.671944 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1425/2834] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:17:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 15:17:57.984294 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264452fe-3f35-48e4-b304-5bc076a97ba1 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:17:57.985897 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-264452fe-3f35-48e4-b304-5bc076a97ba1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:17:58.002055 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264452fe-3f35-48e4-b304-5bc076a97ba1 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:17:58.008791 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b89e99b-0fc4-4050-8e0c-df4f9b0b1d8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:17:58.009875 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b89e99b-0fc4-4050-8e0c-df4f9b0b1d8a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:17:58.010233 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b89e99b-0fc4-4050-8e0c-df4f9b0b1d8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:17:58.010581 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b89e99b-0fc4-4050-8e0c-df4f9b0b1d8a admin admin] PATH_INFO: `/v3/projects/575e35684b064ccfaca662ea8ee3014d` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:17:58.012842 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b89e99b-0fc4-4050-8e0c-df4f9b0b1d8a admin admin] RBAC: Authorizing `identity:delete_project(project_id=575e35684b064ccfaca662ea8ee3014d)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:17:58.015033 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b89e99b-0fc4-4050-8e0c-df4f9b0b1d8a admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:17:58.069627 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-9b89e99b-0fc4-4050-8e0c-df4f9b0b1d8a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '575e35684b064ccfaca662ea8ee3014d', 'request_id': 'req-9b89e99b-0fc4-4050-8e0c-df4f9b0b1d8a'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:17:58.076554 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-9b89e99b-0fc4-4050-8e0c-df4f9b0b1d8a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '575e35684b064ccfaca662ea8ee3014d', 'request_id': 'req-9b89e99b-0fc4-4050-8e0c-df4f9b0b1d8a'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:17:58.083006 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-9b89e99b-0fc4-4050-8e0c-df4f9b0b1d8a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '575e35684b064ccfaca662ea8ee3014d', 'request_id': 'req-9b89e99b-0fc4-4050-8e0c-df4f9b0b1d8a'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:17:58.104739 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-9b89e99b-0fc4-4050-8e0c-df4f9b0b1d8a admin admin] The token cache is being invalidate because project 575e35684b064ccfaca662ea8ee3014d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 15:17:58.105037 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-9b89e99b-0fc4-4050-8e0c-df4f9b0b1d8a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:17:58.105801 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-9b89e99b-0fc4-4050-8e0c-df4f9b0b1d8a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:17:58.107381 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1410/2835] 10.222.0.46 () {58 vars in 1232 bytes} [Thu May 26 15:17:57 2022] DELETE /identity/v3/projects/575e35684b064ccfaca662ea8ee3014d => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:17:58.111485 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0762d6c-7376-45d1-94fe-3d561ef21da3 service neutron] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:17:58.122636 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0762d6c-7376-45d1-94fe-3d561ef21da3 service neutron] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:17:58.186574 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a62e8d-409b-40d9-8651-597d08335b9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:17:58.187459 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a62e8d-409b-40d9-8651-597d08335b9a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:17:58.187735 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a62e8d-409b-40d9-8651-597d08335b9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:17:58.188008 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a62e8d-409b-40d9-8651-597d08335b9a admin admin] PATH_INFO: `/v3/users/9a83869cdbb44e14a81d234047d2ba3f` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:17:58.189304 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a62e8d-409b-40d9-8651-597d08335b9a admin admin] RBAC: Authorizing `identity:delete_user(user_id=9a83869cdbb44e14a81d234047d2ba3f)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:17:58.191243 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19a62e8d-409b-40d9-8651-597d08335b9a admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:17:58.244853 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-19a62e8d-409b-40d9-8651-597d08335b9a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9a83869cdbb44e14a81d234047d2ba3f'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:17:58.254192 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-19a62e8d-409b-40d9-8651-597d08335b9a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9a83869cdbb44e14a81d234047d2ba3f'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:17:58.254952 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-19a62e8d-409b-40d9-8651-597d08335b9a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9a83869cdbb44e14a81d234047d2ba3f'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:17:58.273082 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-19a62e8d-409b-40d9-8651-597d08335b9a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9a83869cdbb44e14a81d234047d2ba3f'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:17:58.281796 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-19a62e8d-409b-40d9-8651-597d08335b9a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9a83869cdbb44e14a81d234047d2ba3f'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:17:58.284188 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:17:58.289392 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1426/2836] 10.222.0.46 () {58 vars in 1223 bytes} [Thu May 26 15:17:58 2022] DELETE /identity/v3/users/9a83869cdbb44e14a81d234047d2ba3f => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:17:58.614360 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b89e99b-0fc4-4050-8e0c-df4f9b0b1d8a admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:17:58.616669 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-9b89e99b-0fc4-4050-8e0c-df4f9b0b1d8a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:17:58.644326 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b89e99b-0fc4-4050-8e0c-df4f9b0b1d8a admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:17:58.726972 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfc19ac-6a17-4aa8-9b2a-ed4f7dbfbe5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:17:58.727872 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfc19ac-6a17-4aa8-9b2a-ed4f7dbfbe5a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:17:58.728382 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfc19ac-6a17-4aa8-9b2a-ed4f7dbfbe5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:17:58.728738 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfc19ac-6a17-4aa8-9b2a-ed4f7dbfbe5a admin admin] PATH_INFO: `/v3/projects/5ebc044d2ba246bd83b2205f7cb7dff6` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:17:58.730221 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfc19ac-6a17-4aa8-9b2a-ed4f7dbfbe5a admin admin] RBAC: Authorizing `identity:delete_project(project_id=5ebc044d2ba246bd83b2205f7cb7dff6)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:17:58.732552 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfc19ac-6a17-4aa8-9b2a-ed4f7dbfbe5a admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:17:58.783343 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-fdfc19ac-6a17-4aa8-9b2a-ed4f7dbfbe5a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5ebc044d2ba246bd83b2205f7cb7dff6', 'request_id': 'req-fdfc19ac-6a17-4aa8-9b2a-ed4f7dbfbe5a'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:17:58.791357 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-fdfc19ac-6a17-4aa8-9b2a-ed4f7dbfbe5a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5ebc044d2ba246bd83b2205f7cb7dff6', 'request_id': 'req-fdfc19ac-6a17-4aa8-9b2a-ed4f7dbfbe5a'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:17:58.804814 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:17:58.807031 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-fdfc19ac-6a17-4aa8-9b2a-ed4f7dbfbe5a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5ebc044d2ba246bd83b2205f7cb7dff6', 'request_id': 'req-fdfc19ac-6a17-4aa8-9b2a-ed4f7dbfbe5a'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:17:58.822356 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-fdfc19ac-6a17-4aa8-9b2a-ed4f7dbfbe5a admin admin] The token cache is being invalidate because project 5ebc044d2ba246bd83b2205f7cb7dff6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 15:17:58.822666 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-fdfc19ac-6a17-4aa8-9b2a-ed4f7dbfbe5a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:17:58.823478 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-fdfc19ac-6a17-4aa8-9b2a-ed4f7dbfbe5a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:17:58.825173 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1411/2837] 10.222.0.46 () {58 vars in 1232 bytes} [Thu May 26 15:17:58 2022] DELETE /identity/v3/projects/5ebc044d2ba246bd83b2205f7cb7dff6 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:17:58.831911 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a62e8d-409b-40d9-8651-597d08335b9a admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:17:58.834081 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-19a62e8d-409b-40d9-8651-597d08335b9a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:17:58.848268 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a62e8d-409b-40d9-8651-597d08335b9a admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:17:58.926610 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b880c52-39c2-4a60-ba2e-dcc709647732 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:17:58.927456 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b880c52-39c2-4a60-ba2e-dcc709647732 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:17:58.927740 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b880c52-39c2-4a60-ba2e-dcc709647732 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:17:58.928045 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b880c52-39c2-4a60-ba2e-dcc709647732 admin admin] PATH_INFO: `/v3/users/a633b3f1025b4f1a9fe0339f1ee12a39` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:17:58.930215 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b880c52-39c2-4a60-ba2e-dcc709647732 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a633b3f1025b4f1a9fe0339f1ee12a39)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:17:58.932294 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b880c52-39c2-4a60-ba2e-dcc709647732 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:17:58.984895 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-0b880c52-39c2-4a60-ba2e-dcc709647732 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a633b3f1025b4f1a9fe0339f1ee12a39'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:17:58.994137 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-0b880c52-39c2-4a60-ba2e-dcc709647732 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a633b3f1025b4f1a9fe0339f1ee12a39'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:17:58.995414 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-0b880c52-39c2-4a60-ba2e-dcc709647732 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a633b3f1025b4f1a9fe0339f1ee12a39'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:17:59.011593 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-0b880c52-39c2-4a60-ba2e-dcc709647732 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a633b3f1025b4f1a9fe0339f1ee12a39'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:17:59.019822 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-0b880c52-39c2-4a60-ba2e-dcc709647732 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a633b3f1025b4f1a9fe0339f1ee12a39'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:17:59.027860 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1427/2838] 10.222.0.46 () {58 vars in 1223 bytes} [Thu May 26 15:17:58 2022] DELETE /identity/v3/users/a633b3f1025b4f1a9fe0339f1ee12a39 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:17:59.327523 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfc19ac-6a17-4aa8-9b2a-ed4f7dbfbe5a admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:17:59.328969 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-fdfc19ac-6a17-4aa8-9b2a-ed4f7dbfbe5a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:17:59.345355 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfc19ac-6a17-4aa8-9b2a-ed4f7dbfbe5a admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:17:59.413141 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041ac658-ce35-4ef4-9584-1768efaa2d54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:17:59.414163 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041ac658-ce35-4ef4-9584-1768efaa2d54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:17:59.414444 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041ac658-ce35-4ef4-9584-1768efaa2d54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:17:59.414721 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041ac658-ce35-4ef4-9584-1768efaa2d54 admin admin] PATH_INFO: `/v3/projects/aa3b0d5f0a204f8598657c33374796fe` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:17:59.416046 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041ac658-ce35-4ef4-9584-1768efaa2d54 admin admin] RBAC: Authorizing `identity:delete_project(project_id=aa3b0d5f0a204f8598657c33374796fe)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:17:59.417928 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041ac658-ce35-4ef4-9584-1768efaa2d54 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:17:59.481057 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-041ac658-ce35-4ef4-9584-1768efaa2d54 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aa3b0d5f0a204f8598657c33374796fe', 'request_id': 'req-041ac658-ce35-4ef4-9584-1768efaa2d54'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:17:59.487993 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-041ac658-ce35-4ef4-9584-1768efaa2d54 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aa3b0d5f0a204f8598657c33374796fe', 'request_id': 'req-041ac658-ce35-4ef4-9584-1768efaa2d54'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:17:59.494542 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-041ac658-ce35-4ef4-9584-1768efaa2d54 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aa3b0d5f0a204f8598657c33374796fe', 'request_id': 'req-041ac658-ce35-4ef4-9584-1768efaa2d54'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:17:59.510890 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-041ac658-ce35-4ef4-9584-1768efaa2d54 admin admin] The token cache is being invalidate because project aa3b0d5f0a204f8598657c33374796fe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 15:17:59.511368 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-041ac658-ce35-4ef4-9584-1768efaa2d54 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:17:59.512498 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-041ac658-ce35-4ef4-9584-1768efaa2d54 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:17:59.515654 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1412/2839] 10.222.0.46 () {58 vars in 1232 bytes} [Thu May 26 15:17:59 2022] DELETE /identity/v3/projects/aa3b0d5f0a204f8598657c33374796fe => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:17:59.523804 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef145937-7900-4b30-a2a8-c1aab81da56d None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:17:59.524503 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef145937-7900-4b30-a2a8-c1aab81da56d None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:17:59.524503 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef145937-7900-4b30-a2a8-c1aab81da56d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:17:59.543246 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-ef145937-7900-4b30-a2a8-c1aab81da56d None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:17:59.544468 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-ef145937-7900-4b30-a2a8-c1aab81da56d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:17:59.624479 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1428/2840] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:17:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:17:59.633513 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c691acbd-efcb-4953-89ff-2cbf87d7aba0 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:17:59.634195 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c691acbd-efcb-4953-89ff-2cbf87d7aba0 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:17:59.634195 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c691acbd-efcb-4953-89ff-2cbf87d7aba0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:17:59.652856 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-c691acbd-efcb-4953-89ff-2cbf87d7aba0 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:17:59.654205 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-c691acbd-efcb-4953-89ff-2cbf87d7aba0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:17:59.662739 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1413/2841] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:17:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:17:59.667154 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef145937-7900-4b30-a2a8-c1aab81da56d None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:17:59.678053 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef145937-7900-4b30-a2a8-c1aab81da56d None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:17:59.683102 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4abbb03-5960-4d3c-8cd3-9d5440df6c39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:17:59.683891 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4abbb03-5960-4d3c-8cd3-9d5440df6c39 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:17:59.684175 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4abbb03-5960-4d3c-8cd3-9d5440df6c39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:17:59.684454 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4abbb03-5960-4d3c-8cd3-9d5440df6c39 admin admin] PATH_INFO: `/v3/domains` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:17:59.685202 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4abbb03-5960-4d3c-8cd3-9d5440df6c39 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80168) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 15:17:59.685556 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4abbb03-5960-4d3c-8cd3-9d5440df6c39 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:17:59.687353 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4abbb03-5960-4d3c-8cd3-9d5440df6c39 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:17:59.702528 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1429/2842] 10.222.0.46 () {58 vars in 1152 bytes} [Thu May 26 15:17:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 15:17:59.706924 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c691acbd-efcb-4953-89ff-2cbf87d7aba0 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:17:59.716243 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c691acbd-efcb-4953-89ff-2cbf87d7aba0 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:17:59.721727 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc0e717-17bd-48ba-8825-f3230d27101f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:17:59.722394 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc0e717-17bd-48ba-8825-f3230d27101f admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:17:59.722678 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc0e717-17bd-48ba-8825-f3230d27101f admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:17:59.722962 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc0e717-17bd-48ba-8825-f3230d27101f admin admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:17:59.723656 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc0e717-17bd-48ba-8825-f3230d27101f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:17:59.726294 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc0e717-17bd-48ba-8825-f3230d27101f admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:17:59.756665 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1414/2843] 10.222.0.46 () {60 vars in 1152 bytes} [Thu May 26 15:17:59 2022] POST /identity/v3/projects => generated 383 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 15:17:59.760907 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4abbb03-5960-4d3c-8cd3-9d5440df6c39 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:17:59.772011 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4abbb03-5960-4d3c-8cd3-9d5440df6c39 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:17:59.776073 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:17:59.835906 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafe5e60-b13d-4c70-80d6-ad4280a54ffe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:17:59.836611 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafe5e60-b13d-4c70-80d6-ad4280a54ffe admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:17:59.836895 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafe5e60-b13d-4c70-80d6-ad4280a54ffe admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:17:59.837175 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafe5e60-b13d-4c70-80d6-ad4280a54ffe admin admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:17:59.837975 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafe5e60-b13d-4c70-80d6-ad4280a54ffe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:17:59.839946 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bafe5e60-b13d-4c70-80d6-ad4280a54ffe admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:17:59.865278 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1430/2844] 10.222.0.46 () {60 vars in 1143 bytes} [Thu May 26 15:17:59 2022] POST /identity/v3/users => generated 292 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 15:17:59.867903 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc0e717-17bd-48ba-8825-f3230d27101f admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:17:59.879650 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc0e717-17bd-48ba-8825-f3230d27101f admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:17:59.886995 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c285e4-a381-4ed6-a402-dde75d2e69f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:17:59.888106 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c285e4-a381-4ed6-a402-dde75d2e69f3 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:17:59.888605 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c285e4-a381-4ed6-a402-dde75d2e69f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:17:59.889104 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c285e4-a381-4ed6-a402-dde75d2e69f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:17:59.890513 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c285e4-a381-4ed6-a402-dde75d2e69f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:17:59.892611 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c285e4-a381-4ed6-a402-dde75d2e69f3 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:17:59.904148 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1415/2845] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 15:17:59 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 15:17:59.908283 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafe5e60-b13d-4c70-80d6-ad4280a54ffe admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:17:59.919067 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafe5e60-b13d-4c70-80d6-ad4280a54ffe admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:17:59.923698 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e9a085-d411-4ea4-b3f7-ed14b81cbcec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:17:59.924363 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e9a085-d411-4ea4-b3f7-ed14b81cbcec admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:17:59.924648 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e9a085-d411-4ea4-b3f7-ed14b81cbcec admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:17:59.924926 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e9a085-d411-4ea4-b3f7-ed14b81cbcec admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:17:59.925691 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e9a085-d411-4ea4-b3f7-ed14b81cbcec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:17:59.928078 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e9a085-d411-4ea4-b3f7-ed14b81cbcec admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:17:59.941556 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1431/2846] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 15:17:59 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 15:17:59.944193 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c285e4-a381-4ed6-a402-dde75d2e69f3 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:17:59.960180 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c285e4-a381-4ed6-a402-dde75d2e69f3 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:17:59.965358 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdd5e51-dea5-4613-9640-9f085486a640 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:17:59.966296 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdd5e51-dea5-4613-9640-9f085486a640 admin admin] REQUEST_METHOD: `PUT` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:17:59.966574 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdd5e51-dea5-4613-9640-9f085486a640 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:17:59.966849 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdd5e51-dea5-4613-9640-9f085486a640 admin admin] PATH_INFO: `/v3/projects/f3013d3be91c42eeb2aa66c738599516/users/0b71674e37974d9e9d22adf85a90ed47/roles/d4088d1845a24fbd9bd8975ce0b1d3d0` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:17:59.983287 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bdd5e51-dea5-4613-9640-9f085486a640 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f3013d3be91c42eeb2aa66c738599516, user_id=0b71674e37974d9e9d22adf85a90ed47, role_id=d4088d1845a24fbd9bd8975ce0b1d3d0)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:17:59.985995 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bdd5e51-dea5-4613-9640-9f085486a640 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:18:00.002179 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1416/2847] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 15:17:59 2022] PUT /identity/v3/projects/f3013d3be91c42eeb2aa66c738599516/users/0b71674e37974d9e9d22adf85a90ed47/roles/d4088d1845a24fbd9bd8975ce0b1d3d0 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:18:00.021082 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fdb4b3-419f-433a-8e38-b448faad29c1 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:18:00.021082 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fdb4b3-419f-433a-8e38-b448faad29c1 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:18:00.021082 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fdb4b3-419f-433a-8e38-b448faad29c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:18:00.061009 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-28fdb4b3-419f-433a-8e38-b448faad29c1 None None] MFA Rules not processed for user `0b71674e37974d9e9d22adf85a90ed47`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:18:00.062301 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-28fdb4b3-419f-433a-8e38-b448faad29c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:18:00.134943 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1432/2848] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:18:00 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:18:00.139819 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdd5e51-dea5-4613-9640-9f085486a640 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:18:00.162154 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=88,keystone:INSERT=4,keystone:DELETE=8 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:18:00.211756 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdd5e51-dea5-4613-9640-9f085486a640 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:18:00.217011 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a3afd7-2277-4b0e-a849-c031cd621ca6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:18:00.217885 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a3afd7-2277-4b0e-a849-c031cd621ca6 admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:18:00.218251 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a3afd7-2277-4b0e-a849-c031cd621ca6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:18:00.218535 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a3afd7-2277-4b0e-a849-c031cd621ca6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:18:00.219228 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a3afd7-2277-4b0e-a849-c031cd621ca6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:18:00.221189 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1a3afd7-2277-4b0e-a849-c031cd621ca6 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:18:00.257745 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1417/2849] 10.222.0.46 () {60 vars in 1152 bytes} [Thu May 26 15:18:00 2022] POST /identity/v3/projects => generated 383 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 15:18:00.261955 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fdb4b3-419f-433a-8e38-b448faad29c1 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:18:00.275166 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fdb4b3-419f-433a-8e38-b448faad29c1 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:18:00.337327 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817a8edb-3272-4f2b-ab0f-fe13e2aa254a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:18:00.338032 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817a8edb-3272-4f2b-ab0f-fe13e2aa254a admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:18:00.338318 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817a8edb-3272-4f2b-ab0f-fe13e2aa254a admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:18:00.338657 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817a8edb-3272-4f2b-ab0f-fe13e2aa254a admin admin] PATH_INFO: `/v3/users` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:18:00.339532 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817a8edb-3272-4f2b-ab0f-fe13e2aa254a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:18:00.341531 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817a8edb-3272-4f2b-ab0f-fe13e2aa254a admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:18:00.363599 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1433/2850] 10.222.0.46 () {60 vars in 1143 bytes} [Thu May 26 15:18:00 2022] POST /identity/v3/users => generated 298 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 15:18:00.369070 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a3afd7-2277-4b0e-a849-c031cd621ca6 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:18:00.379650 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a3afd7-2277-4b0e-a849-c031cd621ca6 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:18:00.385721 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb5ca68-1e4e-4bb9-b970-18deda90c59a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:18:00.386776 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb5ca68-1e4e-4bb9-b970-18deda90c59a admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:18:00.387233 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb5ca68-1e4e-4bb9-b970-18deda90c59a admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:18:00.387692 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb5ca68-1e4e-4bb9-b970-18deda90c59a admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:18:00.388949 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb5ca68-1e4e-4bb9-b970-18deda90c59a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:18:00.391770 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb5ca68-1e4e-4bb9-b970-18deda90c59a admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:18:00.408172 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1418/2851] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 15:18:00 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 15:18:00.410298 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817a8edb-3272-4f2b-ab0f-fe13e2aa254a admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:18:00.417543 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:18:00.422193 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817a8edb-3272-4f2b-ab0f-fe13e2aa254a admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:18:00.429758 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b4a45c-47d6-4148-bb38-18b4d8c1d58b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:18:00.430694 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b4a45c-47d6-4148-bb38-18b4d8c1d58b admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:18:00.430978 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b4a45c-47d6-4148-bb38-18b4d8c1d58b admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:18:00.431258 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b4a45c-47d6-4148-bb38-18b4d8c1d58b admin admin] PATH_INFO: `/v3/projects/a54890b7835a49bd9e44d1cc7dbaf573/users/2614228b883e4541a78985883aa5947b/roles/a2e551773cbd4df084020676865b8222` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:18:00.447856 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b4a45c-47d6-4148-bb38-18b4d8c1d58b admin admin] RBAC: Authorizing `identity:create_grant(project_id=a54890b7835a49bd9e44d1cc7dbaf573, user_id=2614228b883e4541a78985883aa5947b, role_id=a2e551773cbd4df084020676865b8222)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:18:00.450653 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b4a45c-47d6-4148-bb38-18b4d8c1d58b admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:18:00.464289 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1434/2852] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 15:18:00 2022] PUT /identity/v3/projects/a54890b7835a49bd9e44d1cc7dbaf573/users/2614228b883e4541a78985883aa5947b/roles/a2e551773cbd4df084020676865b8222 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:18:00.468477 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb5ca68-1e4e-4bb9-b970-18deda90c59a admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:18:00.480032 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb5ca68-1e4e-4bb9-b970-18deda90c59a admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:18:00.548980 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c060c658-6c35-417a-9c65-2260198aa238 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:18:00.549747 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c060c658-6c35-417a-9c65-2260198aa238 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:18:00.550093 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c060c658-6c35-417a-9c65-2260198aa238 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:18:00.550424 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c060c658-6c35-417a-9c65-2260198aa238 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:18:00.551172 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c060c658-6c35-417a-9c65-2260198aa238 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:18:00.553396 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c060c658-6c35-417a-9c65-2260198aa238 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:18:00.564363 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1419/2853] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 15:18:00 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 15:18:00.568386 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b4a45c-47d6-4148-bb38-18b4d8c1d58b admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:18:00.579358 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b4a45c-47d6-4148-bb38-18b4d8c1d58b admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:18:00.584370 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a269f1-891b-4362-995a-800e255cb35e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:18:00.585288 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a269f1-891b-4362-995a-800e255cb35e admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:18:00.585846 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a269f1-891b-4362-995a-800e255cb35e admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:18:00.586032 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a269f1-891b-4362-995a-800e255cb35e admin admin] PATH_INFO: `/v3/projects/a54890b7835a49bd9e44d1cc7dbaf573/users/2614228b883e4541a78985883aa5947b/roles/a2e551773cbd4df084020676865b8222` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:18:00.588692 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a269f1-891b-4362-995a-800e255cb35e admin admin] RBAC: Authorizing `identity:create_grant(project_id=a54890b7835a49bd9e44d1cc7dbaf573, user_id=2614228b883e4541a78985883aa5947b, role_id=a2e551773cbd4df084020676865b8222)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:18:00.591503 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a269f1-891b-4362-995a-800e255cb35e admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:18:00.605908 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1435/2854] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 15:18:00 2022] PUT /identity/v3/projects/a54890b7835a49bd9e44d1cc7dbaf573/users/2614228b883e4541a78985883aa5947b/roles/a2e551773cbd4df084020676865b8222 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:18:00.622523 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12d3845-d6b6-4f00-8422-025c6e400b32 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:18:00.623210 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12d3845-d6b6-4f00-8422-025c6e400b32 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:18:00.623210 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12d3845-d6b6-4f00-8422-025c6e400b32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:18:00.651906 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-e12d3845-d6b6-4f00-8422-025c6e400b32 None None] MFA Rules not processed for user `2614228b883e4541a78985883aa5947b`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:18:00.653166 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-e12d3845-d6b6-4f00-8422-025c6e400b32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:18:00.729343 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1420/2855] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:18:00 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:18:00.740850 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a269f1-891b-4362-995a-800e255cb35e admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:18:00.743394 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-88a269f1-891b-4362-995a-800e255cb35e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:18:00.811735 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a269f1-891b-4362-995a-800e255cb35e admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:18:00.815354 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c21d0e-45fb-4328-9751-dab6ed023cbd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5b5b35614a547d783e6e77ddb8c1c03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:18:00.816018 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c21d0e-45fb-4328-9751-dab6ed023cbd service cinder] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:18:00.816343 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c21d0e-45fb-4328-9751-dab6ed023cbd service cinder] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:18:00.816631 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c21d0e-45fb-4328-9751-dab6ed023cbd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:18:00.881385 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c21d0e-45fb-4328-9751-dab6ed023cbd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:18:00.882877 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c21d0e-45fb-4328-9751-dab6ed023cbd service cinder] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:18:00.892360 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1436/2856] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:18:00 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 15:18:04.970792 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c4484e-6094-4082-98f9-2d21847544c4 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:18:04.971466 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c4484e-6094-4082-98f9-2d21847544c4 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:18:04.971466 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c4484e-6094-4082-98f9-2d21847544c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:18:04.992526 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-63c4484e-6094-4082-98f9-2d21847544c4 None None] MFA Rules not processed for user `176fec434836494a9af12c96be3a8eb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:18:04.993787 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-63c4484e-6094-4082-98f9-2d21847544c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:18:04.999086 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:18:05.055527 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1421/2857] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:18:04 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:18:05.410336 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c21d0e-45fb-4328-9751-dab6ed023cbd service cinder] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:18:05.420998 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c21d0e-45fb-4328-9751-dab6ed023cbd service cinder] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:18:05.424853 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa161cf0-bf5d-4a7a-be73-29e79c211fb0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5b5b35614a547d783e6e77ddb8c1c03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:18:05.425542 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa161cf0-bf5d-4a7a-be73-29e79c211fb0 service cinder] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:18:05.425828 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa161cf0-bf5d-4a7a-be73-29e79c211fb0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:18:05.426107 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa161cf0-bf5d-4a7a-be73-29e79c211fb0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:18:05.436097 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa161cf0-bf5d-4a7a-be73-29e79c211fb0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:18:05.437616 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa161cf0-bf5d-4a7a-be73-29e79c211fb0 service cinder] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:18:05.449580 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1437/2858] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:18:05 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 15:18:10.455846 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c4484e-6094-4082-98f9-2d21847544c4 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:18:10.533228 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c4484e-6094-4082-98f9-2d21847544c4 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:18:10.537827 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b827edb1-dfc2-4991-ba6c-72ca286b0624 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:18:10.540095 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b827edb1-dfc2-4991-ba6c-72ca286b0624 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:18:10.540095 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b827edb1-dfc2-4991-ba6c-72ca286b0624 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:18:10.540095 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b827edb1-dfc2-4991-ba6c-72ca286b0624 admin admin] PATH_INFO: `/v3/users/0b71674e37974d9e9d22adf85a90ed47` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:18:10.541142 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b827edb1-dfc2-4991-ba6c-72ca286b0624 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0b71674e37974d9e9d22adf85a90ed47)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:18:10.542661 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b827edb1-dfc2-4991-ba6c-72ca286b0624 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:18:10.598090 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-b827edb1-dfc2-4991-ba6c-72ca286b0624 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0b71674e37974d9e9d22adf85a90ed47'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:18:10.609985 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-b827edb1-dfc2-4991-ba6c-72ca286b0624 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0b71674e37974d9e9d22adf85a90ed47'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:18:10.610846 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-b827edb1-dfc2-4991-ba6c-72ca286b0624 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0b71674e37974d9e9d22adf85a90ed47'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:18:10.630335 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-b827edb1-dfc2-4991-ba6c-72ca286b0624 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0b71674e37974d9e9d22adf85a90ed47'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:18:10.637732 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-b827edb1-dfc2-4991-ba6c-72ca286b0624 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0b71674e37974d9e9d22adf85a90ed47'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:18:10.645052 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1422/2859] 10.222.0.46 () {58 vars in 1223 bytes} [Thu May 26 15:18:10 2022] DELETE /identity/v3/users/0b71674e37974d9e9d22adf85a90ed47 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:18:10.651811 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3f8594-d1f0-408f-a564-fb8b54cb7aa4 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:18:10.652440 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3f8594-d1f0-408f-a564-fb8b54cb7aa4 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:18:10.653048 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3f8594-d1f0-408f-a564-fb8b54cb7aa4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:18:10.670896 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-2c3f8594-d1f0-408f-a564-fb8b54cb7aa4 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:18:10.672132 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-2c3f8594-d1f0-408f-a564-fb8b54cb7aa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:18:10.751444 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1438/2860] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:18:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:18:10.764589 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b827edb1-dfc2-4991-ba6c-72ca286b0624 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:18:10.766140 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-b827edb1-dfc2-4991-ba6c-72ca286b0624 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:18:10.811665 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:18:10.833250 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b827edb1-dfc2-4991-ba6c-72ca286b0624 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:18:10.837171 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738fafdf-78a2-4e62-bde1-31894d26ea97 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0156a2f6f3794f479e76b85bdbbe5c26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:18:10.837961 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738fafdf-78a2-4e62-bde1-31894d26ea97 service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:18:10.838186 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738fafdf-78a2-4e62-bde1-31894d26ea97 service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:18:10.838445 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738fafdf-78a2-4e62-bde1-31894d26ea97 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:18:10.848743 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738fafdf-78a2-4e62-bde1-31894d26ea97 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:18:10.850364 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738fafdf-78a2-4e62-bde1-31894d26ea97 service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:18:10.860965 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1423/2861] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:18:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 15:18:11.153771 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3f8594-d1f0-408f-a564-fb8b54cb7aa4 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:18:11.155266 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-2c3f8594-d1f0-408f-a564-fb8b54cb7aa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:18:11.170190 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3f8594-d1f0-408f-a564-fb8b54cb7aa4 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:18:11.175235 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad328c50-0c68-4ae0-a20f-c0aef064cdfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:18:11.176152 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad328c50-0c68-4ae0-a20f-c0aef064cdfe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:18:11.177341 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad328c50-0c68-4ae0-a20f-c0aef064cdfe admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:18:11.177768 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad328c50-0c68-4ae0-a20f-c0aef064cdfe admin admin] PATH_INFO: `/v3/projects/f3013d3be91c42eeb2aa66c738599516` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:18:11.179061 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad328c50-0c68-4ae0-a20f-c0aef064cdfe admin admin] RBAC: Authorizing `identity:delete_project(project_id=f3013d3be91c42eeb2aa66c738599516)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:18:11.180977 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad328c50-0c68-4ae0-a20f-c0aef064cdfe admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:18:11.183783 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:18:11.242283 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-ad328c50-0c68-4ae0-a20f-c0aef064cdfe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f3013d3be91c42eeb2aa66c738599516', 'request_id': 'req-ad328c50-0c68-4ae0-a20f-c0aef064cdfe'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:18:11.251881 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-ad328c50-0c68-4ae0-a20f-c0aef064cdfe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f3013d3be91c42eeb2aa66c738599516', 'request_id': 'req-ad328c50-0c68-4ae0-a20f-c0aef064cdfe'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:18:11.263612 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-ad328c50-0c68-4ae0-a20f-c0aef064cdfe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f3013d3be91c42eeb2aa66c738599516', 'request_id': 'req-ad328c50-0c68-4ae0-a20f-c0aef064cdfe'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:18:11.277131 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-ad328c50-0c68-4ae0-a20f-c0aef064cdfe admin admin] The token cache is being invalidate because project f3013d3be91c42eeb2aa66c738599516 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 15:18:11.277433 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-ad328c50-0c68-4ae0-a20f-c0aef064cdfe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:18:11.278154 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-ad328c50-0c68-4ae0-a20f-c0aef064cdfe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:18:11.279749 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1439/2862] 10.222.0.46 () {58 vars in 1232 bytes} [Thu May 26 15:18:11 2022] DELETE /identity/v3/projects/f3013d3be91c42eeb2aa66c738599516 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:18:11.285338 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738fafdf-78a2-4e62-bde1-31894d26ea97 service neutron] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:18:11.298085 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738fafdf-78a2-4e62-bde1-31894d26ea97 service neutron] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:18:11.382631 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292617d3-a12c-40df-8b86-c581e86d9af9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:18:11.384275 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292617d3-a12c-40df-8b86-c581e86d9af9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:18:11.384498 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292617d3-a12c-40df-8b86-c581e86d9af9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:18:11.384987 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292617d3-a12c-40df-8b86-c581e86d9af9 admin admin] PATH_INFO: `/v3/users/2614228b883e4541a78985883aa5947b` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:18:11.387262 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292617d3-a12c-40df-8b86-c581e86d9af9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2614228b883e4541a78985883aa5947b)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:18:11.389487 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292617d3-a12c-40df-8b86-c581e86d9af9 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:18:11.462828 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-292617d3-a12c-40df-8b86-c581e86d9af9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2614228b883e4541a78985883aa5947b'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:18:11.471380 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-292617d3-a12c-40df-8b86-c581e86d9af9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2614228b883e4541a78985883aa5947b'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:18:11.472284 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-292617d3-a12c-40df-8b86-c581e86d9af9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2614228b883e4541a78985883aa5947b'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:18:11.502940 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-292617d3-a12c-40df-8b86-c581e86d9af9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2614228b883e4541a78985883aa5947b'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:18:11.514081 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-292617d3-a12c-40df-8b86-c581e86d9af9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2614228b883e4541a78985883aa5947b'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:18:11.523078 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1424/2863] 10.222.0.46 () {58 vars in 1223 bytes} [Thu May 26 15:18:11 2022] DELETE /identity/v3/users/2614228b883e4541a78985883aa5947b => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:18:11.857769 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad328c50-0c68-4ae0-a20f-c0aef064cdfe admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:18:11.859392 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-ad328c50-0c68-4ae0-a20f-c0aef064cdfe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:18:11.873813 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad328c50-0c68-4ae0-a20f-c0aef064cdfe admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:18:11.951440 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145b34b4-1b59-4fb1-afe5-44caf79eaaef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:18:11.951440 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145b34b4-1b59-4fb1-afe5-44caf79eaaef admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:18:11.951440 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145b34b4-1b59-4fb1-afe5-44caf79eaaef admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:18:11.951440 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145b34b4-1b59-4fb1-afe5-44caf79eaaef admin admin] PATH_INFO: `/v3/projects/a54890b7835a49bd9e44d1cc7dbaf573` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:18:11.951915 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145b34b4-1b59-4fb1-afe5-44caf79eaaef admin admin] RBAC: Authorizing `identity:delete_project(project_id=a54890b7835a49bd9e44d1cc7dbaf573)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:18:11.953912 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145b34b4-1b59-4fb1-afe5-44caf79eaaef admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:18:12.011802 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-145b34b4-1b59-4fb1-afe5-44caf79eaaef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a54890b7835a49bd9e44d1cc7dbaf573', 'request_id': 'req-145b34b4-1b59-4fb1-afe5-44caf79eaaef'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:18:12.019829 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-145b34b4-1b59-4fb1-afe5-44caf79eaaef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a54890b7835a49bd9e44d1cc7dbaf573', 'request_id': 'req-145b34b4-1b59-4fb1-afe5-44caf79eaaef'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:18:12.026458 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-145b34b4-1b59-4fb1-afe5-44caf79eaaef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a54890b7835a49bd9e44d1cc7dbaf573', 'request_id': 'req-145b34b4-1b59-4fb1-afe5-44caf79eaaef'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:18:12.040742 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-145b34b4-1b59-4fb1-afe5-44caf79eaaef admin admin] The token cache is being invalidate because project a54890b7835a49bd9e44d1cc7dbaf573 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 15:18:12.041092 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-145b34b4-1b59-4fb1-afe5-44caf79eaaef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:18:12.042503 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-145b34b4-1b59-4fb1-afe5-44caf79eaaef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:18:12.044469 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1440/2864] 10.222.0.46 () {58 vars in 1232 bytes} [Thu May 26 15:18:11 2022] DELETE /identity/v3/projects/a54890b7835a49bd9e44d1cc7dbaf573 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:18:12.050764 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc5252f-ec1f-48b0-9ee4-0acbe7dbfb8b None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:18:12.051099 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc5252f-ec1f-48b0-9ee4-0acbe7dbfb8b None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:18:12.051989 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc5252f-ec1f-48b0-9ee4-0acbe7dbfb8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:18:12.075955 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-4dc5252f-ec1f-48b0-9ee4-0acbe7dbfb8b None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:18:12.077772 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-4dc5252f-ec1f-48b0-9ee4-0acbe7dbfb8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:18:12.155168 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:18:12.172312 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1425/2865] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:18:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:18:12.180946 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77885bd7-31cb-4a50-b6ea-20a516325349 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:18:12.181241 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77885bd7-31cb-4a50-b6ea-20a516325349 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:18:12.181779 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77885bd7-31cb-4a50-b6ea-20a516325349 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:18:12.204087 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-77885bd7-31cb-4a50-b6ea-20a516325349 None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:18:12.205660 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-77885bd7-31cb-4a50-b6ea-20a516325349 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:18:12.221487 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc5252f-ec1f-48b0-9ee4-0acbe7dbfb8b None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:18:12.221825 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1441/2866] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:18:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:18:12.234417 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc5252f-ec1f-48b0-9ee4-0acbe7dbfb8b None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:18:12.239964 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823dcf4d-f0a9-4588-a263-fc0b1ac27678 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:18:12.241499 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823dcf4d-f0a9-4588-a263-fc0b1ac27678 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:18:12.242033 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823dcf4d-f0a9-4588-a263-fc0b1ac27678 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:18:12.242648 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823dcf4d-f0a9-4588-a263-fc0b1ac27678 admin admin] PATH_INFO: `/v3/domains` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:18:12.243509 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823dcf4d-f0a9-4588-a263-fc0b1ac27678 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80167) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 26 15:18:12.244099 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823dcf4d-f0a9-4588-a263-fc0b1ac27678 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:18:12.245953 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823dcf4d-f0a9-4588-a263-fc0b1ac27678 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:18:12.262911 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1426/2867] 10.222.0.46 () {58 vars in 1152 bytes} [Thu May 26 15:18:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 26 15:18:12.268310 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77885bd7-31cb-4a50-b6ea-20a516325349 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:18:12.289549 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77885bd7-31cb-4a50-b6ea-20a516325349 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:18:12.295379 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1b362b-6ef2-40bf-9a45-9ac0e1ead675 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:18:12.296124 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1b362b-6ef2-40bf-9a45-9ac0e1ead675 admin admin] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:18:12.296423 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1b362b-6ef2-40bf-9a45-9ac0e1ead675 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:18:12.296706 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1b362b-6ef2-40bf-9a45-9ac0e1ead675 admin admin] PATH_INFO: `/v3/projects` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:18:12.297496 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1b362b-6ef2-40bf-9a45-9ac0e1ead675 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:18:12.299477 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1b362b-6ef2-40bf-9a45-9ac0e1ead675 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:18:12.318328 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:18:12.336358 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1442/2868] 10.222.0.46 () {60 vars in 1152 bytes} [Thu May 26 15:18:12 2022] POST /identity/v3/projects => generated 357 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 15:18:12.340252 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823dcf4d-f0a9-4588-a263-fc0b1ac27678 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:18:12.353100 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823dcf4d-f0a9-4588-a263-fc0b1ac27678 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:18:12.430275 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1696d308-5161-4c62-8d10-b1710ba41c3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:18:12.431645 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1696d308-5161-4c62-8d10-b1710ba41c3a admin admin] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:18:12.432128 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1696d308-5161-4c62-8d10-b1710ba41c3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:18:12.432622 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1696d308-5161-4c62-8d10-b1710ba41c3a admin admin] PATH_INFO: `/v3/users` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:18:12.433742 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1696d308-5161-4c62-8d10-b1710ba41c3a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:18:12.436184 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1696d308-5161-4c62-8d10-b1710ba41c3a admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:18:12.460916 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1427/2869] 10.222.0.46 () {60 vars in 1143 bytes} [Thu May 26 15:18:12 2022] POST /identity/v3/users => generated 279 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 26 15:18:12.468541 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1b362b-6ef2-40bf-9a45-9ac0e1ead675 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:18:12.480268 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1b362b-6ef2-40bf-9a45-9ac0e1ead675 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:18:12.485574 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f337dea-46a9-4366-b7c0-1179a4adca00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:18:12.486370 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f337dea-46a9-4366-b7c0-1179a4adca00 admin admin] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:18:12.486732 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f337dea-46a9-4366-b7c0-1179a4adca00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:18:12.487087 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f337dea-46a9-4366-b7c0-1179a4adca00 admin admin] PATH_INFO: `/v3/roles` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:18:12.487899 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f337dea-46a9-4366-b7c0-1179a4adca00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:18:12.490552 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f337dea-46a9-4366-b7c0-1179a4adca00 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:18:12.505913 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1443/2870] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 15:18:12 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 15:18:12.510097 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1696d308-5161-4c62-8d10-b1710ba41c3a admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:18:12.527258 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1696d308-5161-4c62-8d10-b1710ba41c3a admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:18:12.532469 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271e3a0d-5dee-4cca-be82-1bcf05a06142 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:18:12.533147 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271e3a0d-5dee-4cca-be82-1bcf05a06142 admin admin] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:18:12.533636 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271e3a0d-5dee-4cca-be82-1bcf05a06142 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:18:12.533919 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271e3a0d-5dee-4cca-be82-1bcf05a06142 admin admin] PATH_INFO: `/v3/roles` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:18:12.534663 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271e3a0d-5dee-4cca-be82-1bcf05a06142 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:18:12.536883 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271e3a0d-5dee-4cca-be82-1bcf05a06142 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:18:12.549143 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1428/2871] 10.222.0.46 () {58 vars in 1121 bytes} [Thu May 26 15:18:12 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 26 15:18:12.558568 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f337dea-46a9-4366-b7c0-1179a4adca00 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:18:12.570856 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f337dea-46a9-4366-b7c0-1179a4adca00 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:18:12.578616 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431969c3-f672-41eb-bdd5-2e4c56f1a0c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:18:12.579526 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431969c3-f672-41eb-bdd5-2e4c56f1a0c3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:18:12.579817 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431969c3-f672-41eb-bdd5-2e4c56f1a0c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:18:12.580100 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431969c3-f672-41eb-bdd5-2e4c56f1a0c3 admin admin] PATH_INFO: `/v3/projects/957af46f2d27468f8dbc588771dec7f3/users/27729a502fd14d91a4903573a855399b/roles/d4088d1845a24fbd9bd8975ce0b1d3d0` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:18:12.602852 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431969c3-f672-41eb-bdd5-2e4c56f1a0c3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=957af46f2d27468f8dbc588771dec7f3, user_id=27729a502fd14d91a4903573a855399b, role_id=d4088d1845a24fbd9bd8975ce0b1d3d0)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:18:12.610290 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431969c3-f672-41eb-bdd5-2e4c56f1a0c3 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:18:12.630822 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1444/2872] 10.222.0.46 () {60 vars in 1482 bytes} [Thu May 26 15:18:12 2022] PUT /identity/v3/projects/957af46f2d27468f8dbc588771dec7f3/users/27729a502fd14d91a4903573a855399b/roles/d4088d1845a24fbd9bd8975ce0b1d3d0 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:18:12.643844 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db47a3f-b8b5-4626-9099-f2b8a7e5aa66 None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:18:12.644136 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db47a3f-b8b5-4626-9099-f2b8a7e5aa66 None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:18:12.644418 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db47a3f-b8b5-4626-9099-f2b8a7e5aa66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:18:12.675754 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-6db47a3f-b8b5-4626-9099-f2b8a7e5aa66 None None] MFA Rules not processed for user `27729a502fd14d91a4903573a855399b`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:18:12.677573 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-6db47a3f-b8b5-4626-9099-f2b8a7e5aa66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:18:12.755895 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1429/2873] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:18:12 2022] POST /identity/v3/auth/tokens => generated 3224 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:18:12.794705 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431969c3-f672-41eb-bdd5-2e4c56f1a0c3 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:18:12.796795 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-431969c3-f672-41eb-bdd5-2e4c56f1a0c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:18:12.875032 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431969c3-f672-41eb-bdd5-2e4c56f1a0c3 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:18:12.878769 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440e333c-e09c-4aeb-a028-e336f216f929 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd5b5b35614a547d783e6e77ddb8c1c03', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:18:12.880011 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440e333c-e09c-4aeb-a028-e336f216f929 service cinder] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:18:12.880290 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440e333c-e09c-4aeb-a028-e336f216f929 service cinder] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:18:12.880570 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440e333c-e09c-4aeb-a028-e336f216f929 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:18:12.889840 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440e333c-e09c-4aeb-a028-e336f216f929 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:18:12.891315 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440e333c-e09c-4aeb-a028-e336f216f929 service cinder] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:18:12.902784 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1445/2874] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:18:12 2022] GET /identity/v3/auth/tokens => generated 3224 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 15:18:12.927834 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db47a3f-b8b5-4626-9099-f2b8a7e5aa66 None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:18:12.993006 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:18:13.006972 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db47a3f-b8b5-4626-9099-f2b8a7e5aa66 None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:18:13.012694 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edca824-1a40-4639-a290-18520c8b563d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:18:13.013944 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edca824-1a40-4639-a290-18520c8b563d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:18:13.014244 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edca824-1a40-4639-a290-18520c8b563d admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:18:13.014513 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2edca824-1a40-4639-a290-18520c8b563d admin admin] PATH_INFO: `/v3/users/27729a502fd14d91a4903573a855399b` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:18:13.016453 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edca824-1a40-4639-a290-18520c8b563d admin admin] RBAC: Authorizing `identity:delete_user(user_id=27729a502fd14d91a4903573a855399b)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:18:13.019528 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2edca824-1a40-4639-a290-18520c8b563d admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:18:13.089210 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-2edca824-1a40-4639-a290-18520c8b563d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '27729a502fd14d91a4903573a855399b'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:18:13.099969 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-2edca824-1a40-4639-a290-18520c8b563d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '27729a502fd14d91a4903573a855399b'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:18:13.101013 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-2edca824-1a40-4639-a290-18520c8b563d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '27729a502fd14d91a4903573a855399b'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:18:13.121935 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-2edca824-1a40-4639-a290-18520c8b563d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '27729a502fd14d91a4903573a855399b'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:18:13.132080 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-2edca824-1a40-4639-a290-18520c8b563d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '27729a502fd14d91a4903573a855399b'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:18:13.150761 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1430/2875] 10.222.0.46 () {58 vars in 1223 bytes} [Thu May 26 15:18:12 2022] DELETE /identity/v3/users/27729a502fd14d91a4903573a855399b => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:18:13.155101 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da68ba7-eacb-46fe-894c-932890be725f None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:18:13.155797 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da68ba7-eacb-46fe-894c-932890be725f None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:18:13.156315 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da68ba7-eacb-46fe-894c-932890be725f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:18:13.186820 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-0da68ba7-eacb-46fe-894c-932890be725f None None] MFA Rules not processed for user `dede0ef3af5b453796bce68732b5fd29`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:18:13.187983 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-0da68ba7-eacb-46fe-894c-932890be725f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:18:13.281948 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1446/2876] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:18:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:18:13.301614 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edca824-1a40-4639-a290-18520c8b563d admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:18:13.303155 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-2edca824-1a40-4639-a290-18520c8b563d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:18:13.404749 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2edca824-1a40-4639-a290-18520c8b563d admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:18:13.408574 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cac382a-d02e-4e34-8d71-acdf0db22474 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0156a2f6f3794f479e76b85bdbbe5c26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:18:13.409477 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cac382a-d02e-4e34-8d71-acdf0db22474 service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:18:13.409772 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cac382a-d02e-4e34-8d71-acdf0db22474 service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:18:13.410208 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cac382a-d02e-4e34-8d71-acdf0db22474 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:18:13.432952 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cac382a-d02e-4e34-8d71-acdf0db22474 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:18:13.434632 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cac382a-d02e-4e34-8d71-acdf0db22474 service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:18:13.448110 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1431/2877] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:18:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 15:18:13.918444 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da68ba7-eacb-46fe-894c-932890be725f None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:18:13.921107 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-0da68ba7-eacb-46fe-894c-932890be725f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:18:13.937416 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da68ba7-eacb-46fe-894c-932890be725f None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:18:13.944278 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb08c24-ad57-4235-b0cd-5094c5d0f226 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:18:13.945195 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb08c24-ad57-4235-b0cd-5094c5d0f226 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:18:13.945444 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb08c24-ad57-4235-b0cd-5094c5d0f226 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:18:13.945753 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb08c24-ad57-4235-b0cd-5094c5d0f226 admin admin] PATH_INFO: `/v3/projects/957af46f2d27468f8dbc588771dec7f3` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:18:13.946973 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb08c24-ad57-4235-b0cd-5094c5d0f226 admin admin] RBAC: Authorizing `identity:delete_project(project_id=957af46f2d27468f8dbc588771dec7f3)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:18:13.950097 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb08c24-ad57-4235-b0cd-5094c5d0f226 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:18:14.006865 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-6eb08c24-ad57-4235-b0cd-5094c5d0f226 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '957af46f2d27468f8dbc588771dec7f3', 'request_id': 'req-6eb08c24-ad57-4235-b0cd-5094c5d0f226'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:18:14.015065 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-6eb08c24-ad57-4235-b0cd-5094c5d0f226 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '957af46f2d27468f8dbc588771dec7f3', 'request_id': 'req-6eb08c24-ad57-4235-b0cd-5094c5d0f226'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:18:14.016437 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:INSERT=3,keystone:SELECT=92,keystone:DELETE=5 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:18:14.022605 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-6eb08c24-ad57-4235-b0cd-5094c5d0f226 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '957af46f2d27468f8dbc588771dec7f3', 'request_id': 'req-6eb08c24-ad57-4235-b0cd-5094c5d0f226'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:18:14.035285 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-6eb08c24-ad57-4235-b0cd-5094c5d0f226 admin admin] The token cache is being invalidate because project 957af46f2d27468f8dbc588771dec7f3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 15:18:14.036123 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-6eb08c24-ad57-4235-b0cd-5094c5d0f226 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:18:14.036788 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-6eb08c24-ad57-4235-b0cd-5094c5d0f226 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:18:14.038365 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1447/2878] 10.222.0.46 () {58 vars in 1232 bytes} [Thu May 26 15:18:13 2022] DELETE /identity/v3/projects/957af46f2d27468f8dbc588771dec7f3 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:18:23.441629 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=55,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:18:24.031931 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:DELETE=2,keystone:SELECT=5 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:18:26.643405 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cac382a-d02e-4e34-8d71-acdf0db22474 service neutron] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:18:26.652634 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cac382a-d02e-4e34-8d71-acdf0db22474 service neutron] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:18:26.705502 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d00445-d9e2-4b63-91b3-e8663c877fbc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0156a2f6f3794f479e76b85bdbbe5c26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:18:26.706162 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d00445-d9e2-4b63-91b3-e8663c877fbc service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:18:26.706443 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d00445-d9e2-4b63-91b3-e8663c877fbc service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:18:26.706720 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d00445-d9e2-4b63-91b3-e8663c877fbc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:18:26.708534 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-c2d00445-d9e2-4b63-91b3-e8663c877fbc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:18:26.783400 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d00445-d9e2-4b63-91b3-e8663c877fbc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:18:26.784862 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d00445-d9e2-4b63-91b3-e8663c877fbc service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:18:26.796095 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1432/2879] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:18:26 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 15:18:36.791270 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=50 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:18:49.541831 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb08c24-ad57-4235-b0cd-5094c5d0f226 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:18:49.543354 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-6eb08c24-ad57-4235-b0cd-5094c5d0f226 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:18:49.560265 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb08c24-ad57-4235-b0cd-5094c5d0f226 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:18:49.563877 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7d2654-394f-4248-9446-7d5fcca74e98 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0156a2f6f3794f479e76b85bdbbe5c26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:18:49.564528 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7d2654-394f-4248-9446-7d5fcca74e98 service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:18:49.564818 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7d2654-394f-4248-9446-7d5fcca74e98 service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:18:49.565094 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7d2654-394f-4248-9446-7d5fcca74e98 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:18:49.567215 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-ba7d2654-394f-4248-9446-7d5fcca74e98 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:18:49.648826 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7d2654-394f-4248-9446-7d5fcca74e98 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:18:49.650308 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7d2654-394f-4248-9446-7d5fcca74e98 service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:18:49.670296 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1448/2880] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:18:49 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 15:18:55.855082 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d00445-d9e2-4b63-91b3-e8663c877fbc service neutron] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:18:55.864350 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d00445-d9e2-4b63-91b3-e8663c877fbc service neutron] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:18:55.868029 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4596d450-009a-48c7-914c-cf1b1d0f8f19 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0156a2f6f3794f479e76b85bdbbe5c26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:18:55.868683 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4596d450-009a-48c7-914c-cf1b1d0f8f19 service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:18:55.868962 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4596d450-009a-48c7-914c-cf1b1d0f8f19 service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:18:55.869238 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4596d450-009a-48c7-914c-cf1b1d0f8f19 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:18:55.871024 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-4596d450-009a-48c7-914c-cf1b1d0f8f19 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:18:55.885972 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4596d450-009a-48c7-914c-cf1b1d0f8f19 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:18:55.887434 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4596d450-009a-48c7-914c-cf1b1d0f8f19 service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:18:55.899990 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1433/2881] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:18:55 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 15:18:58.251356 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710d43c6-3d33-4b4a-aa8c-792581f68f74 None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:18:58.251863 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710d43c6-3d33-4b4a-aa8c-792581f68f74 None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:18:58.252171 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710d43c6-3d33-4b4a-aa8c-792581f68f74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:18:58.271546 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-710d43c6-3d33-4b4a-aa8c-792581f68f74 None None] MFA Rules not processed for user `8028c520d93449a6a087c281280ee13d`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:18:58.273741 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-710d43c6-3d33-4b4a-aa8c-792581f68f74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:18:58.344952 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1449/2882] 10.222.0.46 () {58 vars in 967 bytes} [Thu May 26 15:18:58 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:18:58.390456 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4596d450-009a-48c7-914c-cf1b1d0f8f19 service neutron] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:18:58.391875 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-4596d450-009a-48c7-914c-cf1b1d0f8f19 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:18:58.466439 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4596d450-009a-48c7-914c-cf1b1d0f8f19 service neutron] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:18:58.471670 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0766f5b4-ec80-4afd-84b4-eb9f0b3fb684 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a4bea272624b45d8ac2ad27bd07df885', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:18:58.472328 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0766f5b4-ec80-4afd-84b4-eb9f0b3fb684 service swift] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:18:58.472609 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0766f5b4-ec80-4afd-84b4-eb9f0b3fb684 service swift] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:18:58.472888 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0766f5b4-ec80-4afd-84b4-eb9f0b3fb684 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:18:58.482538 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0766f5b4-ec80-4afd-84b4-eb9f0b3fb684 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:18:58.484167 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0766f5b4-ec80-4afd-84b4-eb9f0b3fb684 service swift] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:18:58.493877 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1434/2883] 10.222.0.46 () {60 vars in 1384 bytes} [Thu May 26 15:18:58 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 26 15:19:05.511293 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710d43c6-3d33-4b4a-aa8c-792581f68f74 None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:19:05.513043 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-710d43c6-3d33-4b4a-aa8c-792581f68f74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:19:05.614226 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710d43c6-3d33-4b4a-aa8c-792581f68f74 None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:19:05.619597 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921cee73-f18c-46ca-bd49-32a0d79b0029 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:19:05.620493 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921cee73-f18c-46ca-bd49-32a0d79b0029 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:19:05.620778 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921cee73-f18c-46ca-bd49-32a0d79b0029 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:19:05.621163 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921cee73-f18c-46ca-bd49-32a0d79b0029 admin admin] PATH_INFO: `/v3/users/e6e2622df88f4e1b9b81097d6e558f21` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:19:05.622559 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921cee73-f18c-46ca-bd49-32a0d79b0029 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e6e2622df88f4e1b9b81097d6e558f21)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:19:05.624506 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921cee73-f18c-46ca-bd49-32a0d79b0029 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:19:05.666797 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:19:05.683487 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-921cee73-f18c-46ca-bd49-32a0d79b0029 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e6e2622df88f4e1b9b81097d6e558f21'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:19:05.694177 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-921cee73-f18c-46ca-bd49-32a0d79b0029 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e6e2622df88f4e1b9b81097d6e558f21'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:19:05.694788 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-921cee73-f18c-46ca-bd49-32a0d79b0029 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e6e2622df88f4e1b9b81097d6e558f21'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:19:05.706891 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-921cee73-f18c-46ca-bd49-32a0d79b0029 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e6e2622df88f4e1b9b81097d6e558f21'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:19:05.713883 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-921cee73-f18c-46ca-bd49-32a0d79b0029 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e6e2622df88f4e1b9b81097d6e558f21'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:19:05.720068 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1450/2884] 10.222.0.46 () {58 vars in 1223 bytes} [Thu May 26 15:19:05 2022] DELETE /identity/v3/users/e6e2622df88f4e1b9b81097d6e558f21 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:19:05.924486 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0766f5b4-ec80-4afd-84b4-eb9f0b3fb684 service swift] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:19:05.936514 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0766f5b4-ec80-4afd-84b4-eb9f0b3fb684 service swift] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:19:05.999170 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74610173-0074-465f-8ef9-20d02a7b735c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:19:06.000165 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74610173-0074-465f-8ef9-20d02a7b735c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:19:06.000448 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74610173-0074-465f-8ef9-20d02a7b735c admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:19:06.000727 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74610173-0074-465f-8ef9-20d02a7b735c admin admin] PATH_INFO: `/v3/projects/fb7e58e0c6954bb5ae0fafee4c7a6724` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:19:06.001960 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74610173-0074-465f-8ef9-20d02a7b735c admin admin] RBAC: Authorizing `identity:delete_project(project_id=fb7e58e0c6954bb5ae0fafee4c7a6724)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:19:06.003864 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74610173-0074-465f-8ef9-20d02a7b735c admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:19:06.054688 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-74610173-0074-465f-8ef9-20d02a7b735c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fb7e58e0c6954bb5ae0fafee4c7a6724', 'request_id': 'req-74610173-0074-465f-8ef9-20d02a7b735c'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:19:06.061186 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-74610173-0074-465f-8ef9-20d02a7b735c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fb7e58e0c6954bb5ae0fafee4c7a6724', 'request_id': 'req-74610173-0074-465f-8ef9-20d02a7b735c'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:19:06.066893 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-74610173-0074-465f-8ef9-20d02a7b735c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fb7e58e0c6954bb5ae0fafee4c7a6724', 'request_id': 'req-74610173-0074-465f-8ef9-20d02a7b735c'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:19:06.080496 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-74610173-0074-465f-8ef9-20d02a7b735c admin admin] The token cache is being invalidate because project fb7e58e0c6954bb5ae0fafee4c7a6724 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 15:19:06.080774 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-74610173-0074-465f-8ef9-20d02a7b735c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:19:06.081408 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-74610173-0074-465f-8ef9-20d02a7b735c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:19:06.083031 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1435/2885] 10.222.0.46 () {58 vars in 1232 bytes} [Thu May 26 15:19:05 2022] DELETE /identity/v3/projects/fb7e58e0c6954bb5ae0fafee4c7a6724 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:19:06.087785 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921cee73-f18c-46ca-bd49-32a0d79b0029 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:19:06.089254 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-921cee73-f18c-46ca-bd49-32a0d79b0029 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:19:06.109642 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921cee73-f18c-46ca-bd49-32a0d79b0029 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:19:06.171505 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9ef230-29b0-4d2e-a8d5-9e26136755da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:19:06.172450 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9ef230-29b0-4d2e-a8d5-9e26136755da admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:19:06.172677 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9ef230-29b0-4d2e-a8d5-9e26136755da admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:19:06.172953 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9ef230-29b0-4d2e-a8d5-9e26136755da admin admin] PATH_INFO: `/v3/users/eb0306b0e4c74acc849fadf4e8a97367` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:19:06.174188 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9ef230-29b0-4d2e-a8d5-9e26136755da admin admin] RBAC: Authorizing `identity:delete_user(user_id=eb0306b0e4c74acc849fadf4e8a97367)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:19:06.176129 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9ef230-29b0-4d2e-a8d5-9e26136755da admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:19:06.229343 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-7e9ef230-29b0-4d2e-a8d5-9e26136755da admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eb0306b0e4c74acc849fadf4e8a97367'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:19:06.238150 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-7e9ef230-29b0-4d2e-a8d5-9e26136755da admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eb0306b0e4c74acc849fadf4e8a97367'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:19:06.239400 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-7e9ef230-29b0-4d2e-a8d5-9e26136755da admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eb0306b0e4c74acc849fadf4e8a97367'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:19:06.251829 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-7e9ef230-29b0-4d2e-a8d5-9e26136755da admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eb0306b0e4c74acc849fadf4e8a97367'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:19:06.260012 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-7e9ef230-29b0-4d2e-a8d5-9e26136755da admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eb0306b0e4c74acc849fadf4e8a97367'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:19:06.266101 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1451/2886] 10.222.0.46 () {58 vars in 1223 bytes} [Thu May 26 15:19:06 2022] DELETE /identity/v3/users/eb0306b0e4c74acc849fadf4e8a97367 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:19:06.458519 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74610173-0074-465f-8ef9-20d02a7b735c admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:19:06.459180 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-74610173-0074-465f-8ef9-20d02a7b735c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:19:06.474462 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74610173-0074-465f-8ef9-20d02a7b735c admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:19:06.487033 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:19:06.555238 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c38c07-09ab-4b44-86c7-6f846ffa627a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:19:06.556118 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c38c07-09ab-4b44-86c7-6f846ffa627a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:19:06.556399 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c38c07-09ab-4b44-86c7-6f846ffa627a admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:19:06.556670 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c38c07-09ab-4b44-86c7-6f846ffa627a admin admin] PATH_INFO: `/v3/projects/b9d682e03d0d4bfbbf21426fa050fca6` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:19:06.558084 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c38c07-09ab-4b44-86c7-6f846ffa627a admin admin] RBAC: Authorizing `identity:delete_project(project_id=b9d682e03d0d4bfbbf21426fa050fca6)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:19:06.560043 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c38c07-09ab-4b44-86c7-6f846ffa627a admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:19:06.614043 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-a7c38c07-09ab-4b44-86c7-6f846ffa627a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b9d682e03d0d4bfbbf21426fa050fca6', 'request_id': 'req-a7c38c07-09ab-4b44-86c7-6f846ffa627a'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:19:06.621392 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-a7c38c07-09ab-4b44-86c7-6f846ffa627a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b9d682e03d0d4bfbbf21426fa050fca6', 'request_id': 'req-a7c38c07-09ab-4b44-86c7-6f846ffa627a'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:19:06.627879 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-a7c38c07-09ab-4b44-86c7-6f846ffa627a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b9d682e03d0d4bfbbf21426fa050fca6', 'request_id': 'req-a7c38c07-09ab-4b44-86c7-6f846ffa627a'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:19:06.652982 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-a7c38c07-09ab-4b44-86c7-6f846ffa627a admin admin] The token cache is being invalidate because project b9d682e03d0d4bfbbf21426fa050fca6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80167) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 15:19:06.653275 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-a7c38c07-09ab-4b44-86c7-6f846ffa627a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:19:06.653964 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-a7c38c07-09ab-4b44-86c7-6f846ffa627a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:19:06.655528 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1436/2887] 10.222.0.46 () {58 vars in 1232 bytes} [Thu May 26 15:19:06 2022] DELETE /identity/v3/projects/b9d682e03d0d4bfbbf21426fa050fca6 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:19:16.262645 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=70,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:19:16.645440 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=48,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:19:57.564962 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9ef230-29b0-4d2e-a8d5-9e26136755da admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:19:57.566471 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-7e9ef230-29b0-4d2e-a8d5-9e26136755da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:19:57.625839 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9ef230-29b0-4d2e-a8d5-9e26136755da admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:19:57.674202 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301aae60-df8e-4e2f-99c1-8f98b5dbcc54 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0156a2f6f3794f479e76b85bdbbe5c26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:19:57.674843 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301aae60-df8e-4e2f-99c1-8f98b5dbcc54 service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:19:57.675131 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301aae60-df8e-4e2f-99c1-8f98b5dbcc54 service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:19:57.675413 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301aae60-df8e-4e2f-99c1-8f98b5dbcc54 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:19:57.677213 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-301aae60-df8e-4e2f-99c1-8f98b5dbcc54 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:19:57.687087 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301aae60-df8e-4e2f-99c1-8f98b5dbcc54 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:19:57.688516 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301aae60-df8e-4e2f-99c1-8f98b5dbcc54 service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:19:57.697502 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1452/2888] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:19:57 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 15:20:07.692964 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=46 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:20:26.859057 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c38c07-09ab-4b44-86c7-6f846ffa627a admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:20:26.860820 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-a7c38c07-09ab-4b44-86c7-6f846ffa627a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:20:26.985623 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c38c07-09ab-4b44-86c7-6f846ffa627a admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:20:26.996711 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc12f6e2-72b4-4d4d-88a4-a7cf73220a86 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6639353c70449d1b3d10cd6487c3b00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:20:26.997423 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc12f6e2-72b4-4d4d-88a4-a7cf73220a86 service nova] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:20:26.997742 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc12f6e2-72b4-4d4d-88a4-a7cf73220a86 service nova] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:20:26.998046 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc12f6e2-72b4-4d4d-88a4-a7cf73220a86 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:20:27.000011 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-dc12f6e2-72b4-4d4d-88a4-a7cf73220a86 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:20:27.023196 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc12f6e2-72b4-4d4d-88a4-a7cf73220a86 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:20:27.027000 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc12f6e2-72b4-4d4d-88a4-a7cf73220a86 service nova] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:20:27.044032 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1437/2889] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:20:26 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 15:20:27.845157 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10bd11f-7bfd-4d1e-8a4c-a1fef3406d5f None None] REQUEST_METHOD: `POST` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:20:27.845938 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10bd11f-7bfd-4d1e-8a4c-a1fef3406d5f None None] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:20:27.845938 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10bd11f-7bfd-4d1e-8a4c-a1fef3406d5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:20:27.864888 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.auth.core [None req-e10bd11f-7bfd-4d1e-8a4c-a1fef3406d5f None None] MFA Rules not processed for user `176fec434836494a9af12c96be3a8eb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80168) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:20:27.866163 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-e10bd11f-7bfd-4d1e-8a4c-a1fef3406d5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:20:27.933383 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1453/2890] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:20:27 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:20:29.027093 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc12f6e2-72b4-4d4d-88a4-a7cf73220a86 service nova] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:20:29.029427 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-dc12f6e2-72b4-4d4d-88a4-a7cf73220a86 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:20:29.111535 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc12f6e2-72b4-4d4d-88a4-a7cf73220a86 service nova] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:20:29.116875 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096ac918-1adb-4398-ab46-7a238c00ebc9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf55a00e3b084aa3b22c25a90e6a773c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:20:29.117584 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096ac918-1adb-4398-ab46-7a238c00ebc9 service placement] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:20:29.117895 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096ac918-1adb-4398-ab46-7a238c00ebc9 service placement] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:20:29.118151 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096ac918-1adb-4398-ab46-7a238c00ebc9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:20:29.120590 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-096ac918-1adb-4398-ab46-7a238c00ebc9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:20:29.133302 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096ac918-1adb-4398-ab46-7a238c00ebc9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:20:29.135020 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096ac918-1adb-4398-ab46-7a238c00ebc9 service placement] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:20:29.145846 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1438/2891] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:20:29 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 15:20:37.926195 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=25 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:20:39.141160 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=66 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:20:40.481353 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10bd11f-7bfd-4d1e-8a4c-a1fef3406d5f None None] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:20:40.551531 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10bd11f-7bfd-4d1e-8a4c-a1fef3406d5f None None] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:20:40.558201 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3327fd64-76e8-4491-b84a-168297af2bfc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6639353c70449d1b3d10cd6487c3b00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:20:40.558904 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3327fd64-76e8-4491-b84a-168297af2bfc service nova] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:20:40.559186 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3327fd64-76e8-4491-b84a-168297af2bfc service nova] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:20:40.559457 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3327fd64-76e8-4491-b84a-168297af2bfc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:20:40.561390 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-3327fd64-76e8-4491-b84a-168297af2bfc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:20:40.596111 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3327fd64-76e8-4491-b84a-168297af2bfc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:20:40.598793 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3327fd64-76e8-4491-b84a-168297af2bfc service nova] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:20:40.617024 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1454/2892] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:20:40 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 15:20:50.608620 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=28 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:22:17.634999 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4847eba0-5007-42b2-b1b1-ee85dc46b99c None None] REQUEST_METHOD: `POST` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:22:17.635518 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4847eba0-5007-42b2-b1b1-ee85dc46b99c None None] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:22:17.635627 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4847eba0-5007-42b2-b1b1-ee85dc46b99c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:22:17.653355 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.auth.core [None req-4847eba0-5007-42b2-b1b1-ee85dc46b99c None None] MFA Rules not processed for user `176fec434836494a9af12c96be3a8eb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80167) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 26 15:22:17.654706 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-4847eba0-5007-42b2-b1b1-ee85dc46b99c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:22:17.661869 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1439/2893] 10.222.0.46 () {58 vars in 957 bytes} [Thu May 26 15:22:17 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 26 15:22:17.994896 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3327fd64-76e8-4491-b84a-168297af2bfc service nova] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:22:17.996416 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-3327fd64-76e8-4491-b84a-168297af2bfc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:22:18.030586 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3327fd64-76e8-4491-b84a-168297af2bfc service nova] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:22:18.034378 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13657340-2aaa-479c-a59a-c4a37e979c06 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0156a2f6f3794f479e76b85bdbbe5c26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:22:18.035037 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13657340-2aaa-479c-a59a-c4a37e979c06 service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:22:18.035349 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13657340-2aaa-479c-a59a-c4a37e979c06 service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:22:18.035644 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13657340-2aaa-479c-a59a-c4a37e979c06 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:22:18.037478 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-13657340-2aaa-479c-a59a-c4a37e979c06 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:22:18.049414 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13657340-2aaa-479c-a59a-c4a37e979c06 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:22:18.051210 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13657340-2aaa-479c-a59a-c4a37e979c06 service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:22:18.064444 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1455/2894] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:22:17 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 15:22:27.648156 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=3 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:22:28.059297 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=15 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:22:57.896918 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4847eba0-5007-42b2-b1b1-ee85dc46b99c None None] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:22:57.918794 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4847eba0-5007-42b2-b1b1-ee85dc46b99c None None] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:22:57.924477 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e707d87-4eca-4dd7-add1-0b5047650cd6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf55a00e3b084aa3b22c25a90e6a773c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:22:57.925118 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e707d87-4eca-4dd7-add1-0b5047650cd6 service placement] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:22:57.925404 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e707d87-4eca-4dd7-add1-0b5047650cd6 service placement] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:22:57.925722 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e707d87-4eca-4dd7-add1-0b5047650cd6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:22:57.927558 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-0e707d87-4eca-4dd7-add1-0b5047650cd6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:22:57.952639 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e707d87-4eca-4dd7-add1-0b5047650cd6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:22:57.954271 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e707d87-4eca-4dd7-add1-0b5047650cd6 service placement] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:22:57.964212 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1440/2895] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:22:57 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 15:23:07.959843 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=15 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:24:04.686908 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13657340-2aaa-479c-a59a-c4a37e979c06 service neutron] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:24:04.688321 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-13657340-2aaa-479c-a59a-c4a37e979c06 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:24:04.700712 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13657340-2aaa-479c-a59a-c4a37e979c06 service neutron] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:24:04.704432 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df811188-c9ee-4c74-ad5a-a67bac768ca1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0156a2f6f3794f479e76b85bdbbe5c26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:24:04.705080 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df811188-c9ee-4c74-ad5a-a67bac768ca1 service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:24:04.705387 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df811188-c9ee-4c74-ad5a-a67bac768ca1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:24:04.705717 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df811188-c9ee-4c74-ad5a-a67bac768ca1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:24:04.707502 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-df811188-c9ee-4c74-ad5a-a67bac768ca1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:24:04.720090 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df811188-c9ee-4c74-ad5a-a67bac768ca1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:24:04.721556 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df811188-c9ee-4c74-ad5a-a67bac768ca1 service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:24:04.730636 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1456/2896] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:24:04 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 15:24:05.848376 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e707d87-4eca-4dd7-add1-0b5047650cd6 service placement] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:24:05.849533 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-0e707d87-4eca-4dd7-add1-0b5047650cd6 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:24:05.898183 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e707d87-4eca-4dd7-add1-0b5047650cd6 service placement] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:24:05.904721 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aebae27-1d75-46af-b754-a8acc10fa45c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd6639353c70449d1b3d10cd6487c3b00', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['member', 'service', 'admin', 'reader', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:24:05.905808 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aebae27-1d75-46af-b754-a8acc10fa45c service nova] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:24:05.906284 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aebae27-1d75-46af-b754-a8acc10fa45c service nova] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:24:05.906569 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aebae27-1d75-46af-b754-a8acc10fa45c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:24:05.908355 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-5aebae27-1d75-46af-b754-a8acc10fa45c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:24:05.922981 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aebae27-1d75-46af-b754-a8acc10fa45c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:24:05.924639 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aebae27-1d75-46af-b754-a8acc10fa45c service nova] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:24:05.936687 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1441/2897] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:24:05 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 15:24:06.030309 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df811188-c9ee-4c74-ad5a-a67bac768ca1 service neutron] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:24:06.038773 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df811188-c9ee-4c74-ad5a-a67bac768ca1 service neutron] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:24:06.043006 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c2cb1b-b1ee-4b2c-8ec9-66e56eb99d24 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0156a2f6f3794f479e76b85bdbbe5c26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:24:06.043654 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c2cb1b-b1ee-4b2c-8ec9-66e56eb99d24 service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:24:06.043954 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c2cb1b-b1ee-4b2c-8ec9-66e56eb99d24 service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:24:06.044218 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c2cb1b-b1ee-4b2c-8ec9-66e56eb99d24 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:24:06.046143 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-04c2cb1b-b1ee-4b2c-8ec9-66e56eb99d24 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:24:06.060371 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c2cb1b-b1ee-4b2c-8ec9-66e56eb99d24 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:24:06.062108 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c2cb1b-b1ee-4b2c-8ec9-66e56eb99d24 service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:24:06.073502 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1457/2898] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:24:06 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 15:24:07.251120 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aebae27-1d75-46af-b754-a8acc10fa45c service nova] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:24:07.260641 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aebae27-1d75-46af-b754-a8acc10fa45c service nova] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:24:07.264271 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a362bb-ffc1-4feb-9aaa-c5a5b00c299f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0156a2f6f3794f479e76b85bdbbe5c26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:24:07.264902 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a362bb-ffc1-4feb-9aaa-c5a5b00c299f service neutron] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:24:07.265183 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a362bb-ffc1-4feb-9aaa-c5a5b00c299f service neutron] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:24:07.265483 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a362bb-ffc1-4feb-9aaa-c5a5b00c299f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:24:07.267230 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-82a362bb-ffc1-4feb-9aaa-c5a5b00c299f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:24:07.281073 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a362bb-ffc1-4feb-9aaa-c5a5b00c299f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:24:07.282542 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a362bb-ffc1-4feb-9aaa-c5a5b00c299f service neutron] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:24:07.294034 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1442/2899] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:24:07 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 15:24:16.068417 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=12 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:24:17.288447 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=24 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:24:19.203167 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c2cb1b-b1ee-4b2c-8ec9-66e56eb99d24 service neutron] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:24:19.211989 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c2cb1b-b1ee-4b2c-8ec9-66e56eb99d24 service neutron] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:24:19.215679 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd25262-1297-4c0d-bff8-7c162cb49713 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '0156a2f6f3794f479e76b85bdbbe5c26', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:24:19.216315 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd25262-1297-4c0d-bff8-7c162cb49713 service neutron] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:24:19.216599 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd25262-1297-4c0d-bff8-7c162cb49713 service neutron] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:24:19.216875 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd25262-1297-4c0d-bff8-7c162cb49713 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:24:19.218648 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-bbd25262-1297-4c0d-bff8-7c162cb49713 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:24:19.308041 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd25262-1297-4c0d-bff8-7c162cb49713 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:24:19.309493 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd25262-1297-4c0d-bff8-7c162cb49713 service neutron] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:24:19.328233 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1458/2900] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:24:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 15:24:22.394739 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a362bb-ffc1-4feb-9aaa-c5a5b00c299f service neutron] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:24:22.406204 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a362bb-ffc1-4feb-9aaa-c5a5b00c299f service neutron] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:24:22.411416 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf07e354-5d93-4f3b-9d08-12eb532f85ae service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf55a00e3b084aa3b22c25a90e6a773c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:24:22.412093 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf07e354-5d93-4f3b-9d08-12eb532f85ae service placement] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:24:22.412386 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf07e354-5d93-4f3b-9d08-12eb532f85ae service placement] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:24:22.412669 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf07e354-5d93-4f3b-9d08-12eb532f85ae service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:24:22.414503 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-cf07e354-5d93-4f3b-9d08-12eb532f85ae service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:24:22.427919 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf07e354-5d93-4f3b-9d08-12eb532f85ae service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:24:22.427919 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf07e354-5d93-4f3b-9d08-12eb532f85ae service placement] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:24:22.439008 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1443/2901] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:24:22 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 26 15:24:25.496759 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd25262-1297-4c0d-bff8-7c162cb49713 service neutron] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:24:25.509953 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd25262-1297-4c0d-bff8-7c162cb49713 service neutron] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:24:25.516180 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7418827e-3f1b-495a-b8f1-cead4aa0c853 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf55a00e3b084aa3b22c25a90e6a773c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:24:25.516841 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7418827e-3f1b-495a-b8f1-cead4aa0c853 service placement] REQUEST_METHOD: `GET` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:24:25.517128 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7418827e-3f1b-495a-b8f1-cead4aa0c853 service placement] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:24:25.517402 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7418827e-3f1b-495a-b8f1-cead4aa0c853 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:24:25.519287 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-7418827e-3f1b-495a-b8f1-cead4aa0c853 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:24:25.532837 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7418827e-3f1b-495a-b8f1-cead4aa0c853 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:24:25.534860 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7418827e-3f1b-495a-b8f1-cead4aa0c853 service placement] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:24:25.550291 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1459/2902] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:24:25 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 15:24:25.564802 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf07e354-5d93-4f3b-9d08-12eb532f85ae service placement] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:24:25.566522 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-cf07e354-5d93-4f3b-9d08-12eb532f85ae service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:24:25.585235 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf07e354-5d93-4f3b-9d08-12eb532f85ae service placement] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:24:25.590701 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0d8d2d-9a16-4110-a6e4-ec2821d3dd55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:24:25.591558 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0d8d2d-9a16-4110-a6e4-ec2821d3dd55 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:24:25.591842 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0d8d2d-9a16-4110-a6e4-ec2821d3dd55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:24:25.592117 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0d8d2d-9a16-4110-a6e4-ec2821d3dd55 admin admin] PATH_INFO: `/v3/users/176fec434836494a9af12c96be3a8eb0` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:24:25.593316 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0d8d2d-9a16-4110-a6e4-ec2821d3dd55 admin admin] RBAC: Authorizing `identity:delete_user(user_id=176fec434836494a9af12c96be3a8eb0)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:24:25.595320 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0d8d2d-9a16-4110-a6e4-ec2821d3dd55 admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:24:25.643640 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-9b0d8d2d-9a16-4110-a6e4-ec2821d3dd55 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '176fec434836494a9af12c96be3a8eb0'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:24:25.650528 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-9b0d8d2d-9a16-4110-a6e4-ec2821d3dd55 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '176fec434836494a9af12c96be3a8eb0'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:24:25.651198 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-9b0d8d2d-9a16-4110-a6e4-ec2821d3dd55 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '176fec434836494a9af12c96be3a8eb0'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:24:25.662633 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-9b0d8d2d-9a16-4110-a6e4-ec2821d3dd55 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '176fec434836494a9af12c96be3a8eb0'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:24:25.669845 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-9b0d8d2d-9a16-4110-a6e4-ec2821d3dd55 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '176fec434836494a9af12c96be3a8eb0'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:24:25.675644 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1444/2903] 10.222.0.46 () {58 vars in 1223 bytes} [Thu May 26 15:24:25 2022] DELETE /identity/v3/users/176fec434836494a9af12c96be3a8eb0 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:24:25.849062 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7418827e-3f1b-495a-b8f1-cead4aa0c853 service placement] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:24:25.860571 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7418827e-3f1b-495a-b8f1-cead4aa0c853 service placement] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:24:25.918449 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9640dd-1e34-4eca-a991-44e474501313 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:24:25.919314 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9640dd-1e34-4eca-a991-44e474501313 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:24:25.919599 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9640dd-1e34-4eca-a991-44e474501313 admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:24:25.919878 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9640dd-1e34-4eca-a991-44e474501313 admin admin] PATH_INFO: `/v3/projects/0b161426dbdc44ff8e52676e1bc21558` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:24:25.921046 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9640dd-1e34-4eca-a991-44e474501313 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0b161426dbdc44ff8e52676e1bc21558)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:24:25.922980 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9640dd-1e34-4eca-a991-44e474501313 admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:24:25.978899 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-2e9640dd-1e34-4eca-a991-44e474501313 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0b161426dbdc44ff8e52676e1bc21558', 'request_id': 'req-2e9640dd-1e34-4eca-a991-44e474501313'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:24:25.983182 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:24:25.987392 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-2e9640dd-1e34-4eca-a991-44e474501313 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0b161426dbdc44ff8e52676e1bc21558', 'request_id': 'req-2e9640dd-1e34-4eca-a991-44e474501313'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:24:25.994420 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-2e9640dd-1e34-4eca-a991-44e474501313 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0b161426dbdc44ff8e52676e1bc21558', 'request_id': 'req-2e9640dd-1e34-4eca-a991-44e474501313'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:24:26.005775 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-2e9640dd-1e34-4eca-a991-44e474501313 admin admin] The token cache is being invalidate because project 0b161426dbdc44ff8e52676e1bc21558 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 15:24:26.006178 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-2e9640dd-1e34-4eca-a991-44e474501313 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:24:26.006826 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-2e9640dd-1e34-4eca-a991-44e474501313 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:24:26.008372 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1460/2904] 10.222.0.46 () {58 vars in 1232 bytes} [Thu May 26 15:24:25 2022] DELETE /identity/v3/projects/0b161426dbdc44ff8e52676e1bc21558 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:24:26.012937 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0d8d2d-9a16-4110-a6e4-ec2821d3dd55 admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:24:26.014449 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-9b0d8d2d-9a16-4110-a6e4-ec2821d3dd55 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:24:26.028699 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0d8d2d-9a16-4110-a6e4-ec2821d3dd55 admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:24:26.089839 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd593b5c-c585-4269-bc16-65444d17ac8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:24:26.090742 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd593b5c-c585-4269-bc16-65444d17ac8c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:24:26.091021 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd593b5c-c585-4269-bc16-65444d17ac8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:24:26.091297 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd593b5c-c585-4269-bc16-65444d17ac8c admin admin] PATH_INFO: `/v3/users/2991d18c077c470eab993644b445bce9` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:24:26.092488 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd593b5c-c585-4269-bc16-65444d17ac8c admin admin] RBAC: Authorizing `identity:delete_user(user_id=2991d18c077c470eab993644b445bce9)` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:24:26.094487 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd593b5c-c585-4269-bc16-65444d17ac8c admin admin] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:24:26.144700 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-bd593b5c-c585-4269-bc16-65444d17ac8c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2991d18c077c470eab993644b445bce9'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:24:26.152494 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-bd593b5c-c585-4269-bc16-65444d17ac8c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2991d18c077c470eab993644b445bce9'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:24:26.153131 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-bd593b5c-c585-4269-bc16-65444d17ac8c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2991d18c077c470eab993644b445bce9'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:24:26.158773 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:24:26.168079 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-bd593b5c-c585-4269-bc16-65444d17ac8c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2991d18c077c470eab993644b445bce9'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:24:26.175246 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.notifications [None req-bd593b5c-c585-4269-bc16-65444d17ac8c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2991d18c077c470eab993644b445bce9'} {{(pid=80167) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:24:26.182158 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1445/2905] 10.222.0.46 () {58 vars in 1223 bytes} [Thu May 26 15:24:26 2022] DELETE /identity/v3/users/2991d18c077c470eab993644b445bce9 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 26 15:24:26.348727 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9640dd-1e34-4eca-a991-44e474501313 admin admin] Authenticating user token {{(pid=80168) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:24:26.350271 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.fernet_utils [None req-2e9640dd-1e34-4eca-a991-44e474501313 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80168) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:24:26.364721 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9640dd-1e34-4eca-a991-44e474501313 admin admin] Validating token access rules against request {{(pid=80168) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:24:26.422959 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebebcd18-3986-4275-9cde-e6a30d9cabef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dede0ef3af5b453796bce68732b5fd29', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '064fc3c7af494c988e44e426f21b1c78', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80168) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:24:26.423820 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebebcd18-3986-4275-9cde-e6a30d9cabef admin admin] REQUEST_METHOD: `DELETE` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:24:26.424128 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebebcd18-3986-4275-9cde-e6a30d9cabef admin admin] SCRIPT_NAME: `/identity` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:24:26.424404 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebebcd18-3986-4275-9cde-e6a30d9cabef admin admin] PATH_INFO: `/v3/projects/4658c39721b44ff9a57416f948913eb6` {{(pid=80168) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:24:26.425618 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebebcd18-3986-4275-9cde-e6a30d9cabef admin admin] RBAC: Authorizing `identity:delete_project(project_id=4658c39721b44ff9a57416f948913eb6)` {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:24:26.427506 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebebcd18-3986-4275-9cde-e6a30d9cabef admin admin] RBAC: Authorization granted {{(pid=80168) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:24:26.473445 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-ebebcd18-3986-4275-9cde-e6a30d9cabef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4658c39721b44ff9a57416f948913eb6', 'request_id': 'req-ebebcd18-3986-4275-9cde-e6a30d9cabef'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:24:26.480734 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-ebebcd18-3986-4275-9cde-e6a30d9cabef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4658c39721b44ff9a57416f948913eb6', 'request_id': 'req-ebebcd18-3986-4275-9cde-e6a30d9cabef'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:24:26.486524 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-ebebcd18-3986-4275-9cde-e6a30d9cabef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4658c39721b44ff9a57416f948913eb6', 'request_id': 'req-ebebcd18-3986-4275-9cde-e6a30d9cabef'} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:24:26.497112 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-ebebcd18-3986-4275-9cde-e6a30d9cabef admin admin] The token cache is being invalidate because project 4658c39721b44ff9a57416f948913eb6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80168) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 26 15:24:26.497752 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-ebebcd18-3986-4275-9cde-e6a30d9cabef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:24:26.498041 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG keystone.notifications [None req-ebebcd18-3986-4275-9cde-e6a30d9cabef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80168) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 26 15:24:26.499565 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: [pid: 80168|app: 0|req: 1461/2906] 10.222.0.46 () {58 vars in 1232 bytes} [Thu May 26 15:24:26 2022] DELETE /identity/v3/projects/4658c39721b44ff9a57416f948913eb6 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 26 15:24:36.178209 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=9,keystone:DELETE=1 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:24:36.494349 dv-cinder-788367-11-iscsi devstack@keystone.service[80168]: DEBUG dbcounter [-] [80168] Writing DB stats keystone:SELECT=57,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80168) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 26 15:28:02.828022 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd593b5c-c585-4269-bc16-65444d17ac8c admin admin] Authenticating user token {{(pid=80167) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 26 15:28:02.829512 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-bd593b5c-c585-4269-bc16-65444d17ac8c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:28:02.899309 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd593b5c-c585-4269-bc16-65444d17ac8c admin admin] Validating token access rules against request {{(pid=80167) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 26 15:28:02.959097 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d63e6e1c-0700-4e72-88ce-1b3c23169d13 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bf55a00e3b084aa3b22c25a90e6a773c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'a17fc8227d4c4da6b2847b350e912ee3', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80167) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 26 15:28:02.959768 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63e6e1c-0700-4e72-88ce-1b3c23169d13 service placement] REQUEST_METHOD: `GET` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 26 15:28:02.960054 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63e6e1c-0700-4e72-88ce-1b3c23169d13 service placement] SCRIPT_NAME: `/identity` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 26 15:28:02.960339 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d63e6e1c-0700-4e72-88ce-1b3c23169d13 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80167) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 26 15:28:02.962227 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.fernet_utils [None req-d63e6e1c-0700-4e72-88ce-1b3c23169d13 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80167) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 26 15:28:02.975114 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63e6e1c-0700-4e72-88ce-1b3c23169d13 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 26 15:28:02.976990 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d63e6e1c-0700-4e72-88ce-1b3c23169d13 service placement] RBAC: Authorization granted {{(pid=80167) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 26 15:28:02.988978 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: [pid: 80167|app: 0|req: 1446/2907] 10.222.0.46 () {60 vars in 1365 bytes} [Thu May 26 15:28:02 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 26 15:28:12.984323 dv-cinder-788367-11-iscsi devstack@keystone.service[80167]: DEBUG dbcounter [-] [80167] Writing DB stats keystone:SELECT=58 {{(pid=80167) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}